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CHAPTER I 
INTRODUCTION 
Background 

In 1984-85, the U. S. Department of Education funded nine 
regional educational laboratories to provide R&D services to every 
area of the United States, including Puerto Rico, the U. S. Virgin 
Islands , and U . S. Pacific Territories . This decision represented 
a new beginning for a program that had been through 19 turbulent 
and controversial years. During that period the program had seen 
major shifts in purposes, government support, and relationship of 
the laboratories to other educational organizations . Of 20 
laboratories funded 1n 1965-66, only seven remained. Frequent 
changes in regional boundaries had been nr>ade, but large areas of 
the country had no laboratory to serve them. 

In 1983 the National Institute of Education (NIE), then the 
Federal unit responsible for the program , decided to continue but 
redesign the program and recompete the contracts.- To address the 
problems previously Identified (Chase, 1968; Campbell, et al , 1975; 
Panel, 1979); and put the program on a firmer footing, NIE 
undertook a massive part ici pat ory pi anning effort . Redrawn 
boundaries identified 10 regions. (Two regions, the Northwest and 
the Pacific Territories, were combined for the first five-year 
period . ) To ensure meaningful competition in each region , up to 
three groups in each region were awarded planning grants prior to 
the laboratory competition . 

ERLC 
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The request for proposal (RFP ) that emerged from the pi ann 1ng 
process represented a careful ly thought-out Federal strategy for 
supporting educational Improvement. Nine labs funded as a result 
of that competltl on have now been 1n operation for two years. The 
present repprt represents one component of a broader program 
evaluat 1on plan Intended to hel p pol Icy makers and program 
stake-holders understand how the new program 1s faring (OERI 1987). 
This study provides a descriptive synthesis or documents and 
reports produced by the laboratories themselves. It will be 
complemented by field studies and policy analyses to be undertaken 
during 1987-89. These program evaluation activities have been 
supplemented by sel f -assessment efforts required of the labs and a 
performance evaluation by external peer review teams at the end of 
the first two years. 

Purposes > Methods . and Limitations > 
This study 1 ooks across the nine 1 abor ator 1es and attempts to 
get a clearer picture of how laboratories view their mission and 
strategy, activities undertaken , and rel atlonshlps establ ished . 1 
What are their similarities and differences? The intent is 
descriptive rather than evaluative, even though this project 1s 
part of an evaluation plan. Perhaps 1t will also make a modest 
contribution to the emergent research field of knowledge 
applications, for which a journal, Knowledge in Society , has just 
been established (Dunn and Holzner 1988). 

The RFP for the lab competition laid out a broad framework for 
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lab operations but did not attempt a detailed specification of 
activities. Indeed, it required that each lab program should 
result from a careful process of Interaction with a wide variety of 
organizations and stakeholders 1n the region to establish needs, 
resources, and opportunities peculiar to each region. 

This study has two major fod of interest; (1) results of the 
process whereby the labs negotiated their role in the region, 
identified regional needs, and established their program; and (2) 
strategies chosen to Implement that role. 

The database for the analysis is limited to documents prepared 
by thrt labs themselves, notably planning, needs sensing, 
sel f -assessment , and annual reports. This places some limitations 
on the Interpretation. There 1s, of course, a human tendency to 
"place one's best foot, forward" when a contractor prepares reports 
for Its funding agency, especially considering the history of 
conflict between the continuing labs and prior funding agencies 
(NIE and the Office of Education). Furthermore, there 1s "many a 
slip twlxt" plan and Implementation. 

However, there is also an upside to this situation. According 
to alternative views of the planning process, planning is a 
"sense-making" activity that more often provides an understanding 
of what has been already done than what is to come (Clark 1980). 
Perhaps the same can be said of progress reports. They may be 
rationalizations 1n the bad sense, but they also may be in the good 
sense-making meaning. Further, because we are dealing with 
professionals for whom the norms of science have some meaning, an 

ERJC v 1 . 
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overly cynical view does not seem warranted. If we understand what 
kinds of documents we are dealing with, these documents should 
conta in useful Information. 

More specifically, core documents analyzed for each lab were: 
o F1 ve-yea r pi an for 1986-90* 

o Governance and organ 1zat1ona 1 status report 

o Plan for FY 1988-90 

o Annual reports for 1986 and 1987 

o Needs sensing report for 1986 

o Self-assessment reports for 1986 and 1987 

The time reference of the present report will vary somewhat 
from section to section. Clear iy the above reports have varying 
time references. The major foCus will be on the status of the 
program at the end of the second contract year, I.e. November 30, 
1987. What 1s the status of the program at that point 1n time? 
However, these are dynamic organizations, and we also are 
Interested 1n how they got where they are. Some reference to 
changes made prior to this point 1n time will be introduced where 
appropriate* Information 1s available. Although the plans for years 
three through five were originally excluded from the scope of this 
study, they represent the culmination of the complex needs sensing, 
self-assessment, and planning process of years one and two and were 
found to be a rich source of Information on lab status and thinking 
as the end of their first two years approached. 

Studying nine labs 1s a form of multlslte qjalitatlve 
policy research (Herrlott and Firestone, 1983). Our principal 

*A11 years 1n this report refer to contract years. 
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method 1s to develop typologies both Inductively and deductively to 

explore the similarities and differences among these nine cases. 

The principal points of reference are the framework provided by the 

lab RFP and the research questions posed in the RFP for this 

project. We are particularly Interested in discovering patterns 

that suggest a fit between certain strategies and tactics on the 

one hand and regional needs and opportunities on the other. 

A leitmotif 1n the analysis will be a concern with how the labs 

cope with divergent problems. In contrast to convergent problems, 

such as arithmetic problems, divergent problems have no specific 

solution; one can only develop a modus operandi for coping 

(Schumacher, 1973). Some of the principal divergent problems faced 

by the labs Include: 

o Accountabll 1„ty vs.* flexibility 
o Independence vs . dependence 
o Prime mover vs. catalyst 

o Need to show Impact at the school level and 

rest r 1ct 1on to 1nd 1 rect strategy . 
o Proactive vs. reactive posture 

In may respects this an Insider's report. It was commissioned 
by OERI as part of a more comprehensive evaluation plan (OERI 
1987). Its primary audiences are OERI personnel, the Laboratory 
Review Panel, lab personnel, and those more generally interested in 
Institutional arrangements for educational Improvement . Acronyms 
have bebn used liberally to avoid cumbersome language and should be 
familiar to most members of these audiences. For those who may 
need to refresh their memories, a glossary of acronyms and 
abbrevlat 1ons has been provi ded as Appendi x C . 
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The Laboratory Mission 

La bora tor 1es can be viewed as an Instrument 1n a Fed era 1 macro- 
strategy for knowledge-based educational Improvement. At the time 
they were conceived 1n the 1960 r s the Federa^ Government had been 
support 1ng a smal 1 program of resea rch grunts , pr 1mar 11 y to 
universities, for about ten years. The un1 verslty-based R&D 
Centers and the Educational Resource Information Center (ERIC) had 
been started Just the year before and were the only other Federal 
Initiatives designed to use new organizational entitles for 
bridging the gap between research and practice. Since that time, 
the number of Federal programs that support "assistance 
Institutions" has grown (Laboratory Review Panel 1987). In 
addition, state-supported entitles like Intermediate service 
agencies have waxed snd waned. In designing the new laboratory 
competition, 1t was necessary to take these changes 1n the 
Infrastructure Into account. 

The Federal perspective on the mission of the laboratories 1s 

discussed 1n some detail 1n the RFP and summarized 1n six 

"statements" and the five- task structure (Appendix A) . Anal ys 1s 

suggests that these can be grouped and summarized as three major 

strategies and two sub-strategies : 

THE LINKAGE STRATEGY 

A. Regional sub-strategy 



o Laboratories serve designated regions (Statement 4) 
o Laboratories have Independent governing boards 
(Statement 5) and develop effective governance, 
management , planning , and evaluation systems 
(Task 1). 

o Laboratories work with and through existing 

organizations to Improve schools and classrooms 
(Task 2). 
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o Laboratories work with state-level decisionmakers 
on school Improvement Issues (Task 3). 
B. National sub-strategy 

o Laboratories are part of a nat ion wide system 
(Statement 6 ) . 

o Laboratories work 1 n col 1 aboratlon with centers 
and with other laboratories on regional and 
national educational problems (Task 5) . 

THE IMPROVEMENT STRATEGY 

o Laboratories focus on school and cl ass room 
Improvement (Statement 1 ) . 

o Laboratories work to create research and develop- 
ment-based resources for school Improvement (Task 
4 ) ; laboratories engage in appl led research and 
development that support Improvement (Statement 3) . 

o Laboratories feature dissemination and assistance 
strategies (Statement 2 ) . 

THE SUBSTANTIVE NEED OR ISSUE STRATEGY (EDUCATIONAL FOCUS) 
o Laboratories assess regional needs, capabilities 
and opportunities and establish priorities for 
laboratory activity (Subtask 1.2). 

Each of these tasks and statements 1s discussed 1n some detail 

1n the RFP and in staff papers that were prepared before drafting 

the RFP. 

The nine laboratory plans represent nine Implementations of this 
set of macro-strategies. This study can be viewed largely as a 
description of similarities and differences in ways the nine labs 
have Implemented the Federal concept 1n nine regional contexts. 

The laboratory concept can be summarized graphicall y as a 
three-sided cube (see Figure 1). Each of these strategies can, 1n 
turn, be broken Into a number of levels or tactics. Every lab 
program has these three facets; variation occurs in the approaches 
used to Implement each. A number of typologies and classifications 
( summarized 1n Chapter VI ) are Introduced 1n order to gain an 
understanding of the similarities and differences among labs in 
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or Issue 



Educational 



Focus, Need 



Linkage 



Strategy 




, Improvement Strategy 
Figure 1. Three Dimensions of the Laboratory Mission 

their approaches to these three strategies. This basic schema will 
be used throughout the report to describe lab activities. 



The report 1s organized more or less in terms of the task 
structure set forth 1n the RFP. A chapter on governance and 
organization j>f laboratories deals with approaches used to 
establish each lab and to negotiate Its niche 1n the region's 
organizational Infrastructure. A chapter on planning covers the 
extensive system of neeJs sensing, capability assessmant, 
self-assessment, and priority setting required under Task 1 and 
represents the labs* approach to selecting and designing program 
activities. We then examine the actual projects and programs 
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undertaken for Task 3, examining 1n more derail the needs 
addressed , the organ 1 za t 1onal 1 inkages establ 1 shed , and improvement 
strategies used. Tasks 2 and 4 are then examined together because 
the programs designed under both tasks were found to have both 
linkage and improvement dimensions* Task 5, which was largely in a 
planning mode during the period under review, will be dealt with 
only where Task 5 themes intersect with work under Tasks 2-4. 
Finally, some conclusions regarding the status of the program at 
the end of two years and some personal observations will complete 
the report ♦ 
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CHAPTER II 




GOVERNANCE AND ORGANIZATION OF LABORATORIES 




Organizational Context 




Descrl ot 1on of the Educational Infrastructure 




In considering the -ole of labs It 1s necessary to ask where 




they fit in the overall structure of education. Herriott (1980) 




has provided a useful distinction among three kinds of 




organizations : 




o Service Delivery Organizations fSDOs^ concerned 
primarily with the direct instruction of pupils or 
with effective management of that Instruction. 




o Knowledge Produdna Organizations (KPOs ) concerned 
primarily with the conduct of basic or applied research 
on topics of potential relevance to the subfleld of 
service delivery 




o Service Improvement Organizations fSIOs^ concerned 
primarily with linking the other two subflelds. 




The service delivery field 1s a complex hierarchical system 




operating at five levels: classroom, school, local education agency 




(LEA), Intermediate service agency (ISA) - 1n some instances - and 




state education agency (SEA). The 107,200 public and private 




el ementary and secondary school s 1n 1980 give some sense of the 




large numbers Involved (NCES, 1982). 




Several attempts have been made to describe the organizational 




arrangements for knowledge-based educational improvement (Office of 




Education 1969; NIE 1976; Frankel , Sharp, and Blderman 1979). The 




1979 study, which covers both KPOs and SIOs, Identified the 




following numbers of organizations with some kind of RDD&E 




capabl 1 1ty : 




is 
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37 State Education Agencies 
193 Intermediate Service Agencies 
401 Local Education Agencies 
4 23 Col 1 eges and Uni versi 1 1es 
476 Miscellaneous Organizations 



Allowing for multiple R&D units within organizations (e.g., 
multiple centers within a university), 2,420 R&D entities existed 
within these .,530 organizations. 

The dominant Image 1s one of large size and extreme complexity. 
Most of the 2,434 performers did not specialize in this field. 
There was a high degree of concentration, with the 172 largest 
performers (7 percent) accounting for 68 percent of all 
expenditures. Furthermore, 47 percent of the organizations were 
found in seven states. These national data are somewhat out of 
date and do not reflect the considerable attrition that has taken 
place during the f 80. Individual laboratories are charged with 
maintaining current data of this type for their regions (to be 
discussed below), but no attempt has beer, made to standardize data 
collection or compile national aggregates. 

Knowledge production takes place in many varied organizational 
settings. In universities 1t occurs at three levels: the 
1nd1 vidua 1 project , sma 11 inst i tutes of sever a 1 projects and 
professors, and major research centers, such as those funded by 
OERI, with multiple programs. Other settings include profit and 
not-for-profit Independent firms and research and evaluation units 
of LEAs and SEAs . 

Organ 1 zat 1ons attempting to assist SDOs with knowl edge-based 
Improvement strategies Include many of the same organizations as 
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well as school development councils, the National Diffusion Network 
(NDN) and ISAs, teacher centers, and, more recently, a 
prol if era t ion of ad Fed era 1 1 y sponsored units such as 

desegregation technical assistance centers. 
System or Confi pupation? 

Taken toget her , these organizational sectors are often referred 
to as the "educational R&D System 1 * or" knowledge production and 
utilization (KPU) system". The legislation that created NIE 
declared as the policy of the United States to "build an effective 
educational research and devel opment system" ( Publ ic Law 92-318, 
section 405, June 23, 1972). Yet, according to some observers, 
this set of organizations, SDOs , KPOs, and SIOs, does not meet even 
the minimum requirements of a system (Sieber 1975; Spivak and 
Radnor 1979). For example, there is little functional 
specialization among performers and little balance among functions, 
performers, settings, and supply and demand. 

Coming to the same conclusion, Clark and his colleagues decided 

that continued dependence on the terminology of systems was 

dysfunctional and new frameworks and metaphors were necessary to 

understand knowledge-based school improvement (Guba and Clark 1974; 

Clark and Carroll 1980a and 1980b; Clark 1980; Clark 1984). They 

note that KPU is: 

...highly decentralized, consisting of a number of more 
or less independent and co-equal members, who may from 
time to time find it helpful to form temporary alliances 
but who, in the main, retain their Independence, shun 
authority and activity relationships, and engage in as 
many different kinds of KPU activities as seem to be 
needed and feasible for them to maintain their 
self-sufficiency. (Guba and Clark 1974, 45) 
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This view of reality is termed the "conf igurational 
perspective" and 1s analogous to the concept of a community (Guba 
and Clark, 1974). Such order and regularity as are found are the 
result of negotiation and persuasion . 

While this description may be somewhat overstated, it appears 
closer to reality than the systems view. Except for the planning 
and evaluation functions, 1t also appears to be the perspective of 
those who planned the laboratory recompetlt ion . 



The Regional Focus of Laboratories 
Overview of Regionalism 1n Education 

When the laboratories were first conceived, two conflicting 
schools of thought emerged on how they should be organized and 
governed. One was that there should be a small number of elite 
Institutions in intellectual centers, the other there should be a 
relatively large number of such institutions, organized regionally, 
and blanketing the country. For reasons that have never been 
entirely clear (but probably are related to the inherent political 
appeal of a program serving every congressional district), the 
regional approach was adopted, and the laboratories have been 
regional ever since. 

Regional organizations, however, have no direct constitutional 
basis. Regional educational laboratories may have a contractual 
relationship to the Federal government, but they stand completely 
outside of the state/local structure of education. In this sense, 
labs are marginal entities. 
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In a review of regionalism 1n education, Hofler (1979) 
concludes that there are several bases for regionalism : 

o Regions can be viewed as aggregate composites of M local 
elements." This Implies that the regional entity will 
be largely controlled by the perspectives, Interests, 
values, goals, etc. of the constituent elements. 

o Heterogeneity may be as Important as homogeneity, for 
1t provides the possibility of complementarity among 
diverse resources . 

o Regions may be based on cultures of collaboration: 
attitudes and values that Imply an openness to the 
possibility of working together. 



The form, focus, or scope of regionalism also tends to be 
highly time-dependent, being either short- or long-term, and 
varying over time. 

Hofler concluded that, as "In-between organizations," regional 
organizations have a particular opportunity to provide both 
local-national mediation and opportunities for cross-state/local 
linkages. Further, they are less vulnerable to pendulum swings 
between centralization and decentralization. 

Hofler presents four arguments 1n favor of and four against 
organizing on a regional b^sls. 



The case FOR regional organization: 

o Regional approaches can serve a variety of (although 
not all) educational KPU purposes . 

o Regionalism 1s Inherently suited for local/national 
mediation and for cross-state/local linkages. 

o Regionalism seems to be a political requirement for 
continued congressional support. 
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o "There 1s reasonably strong evidence to suggest that 
contextual forces do at times converge to provide a 
basis for successful uses of regionalism" (p. 14). 

The case AGAINST regional organization : 

o Regionalism seems to have no preemptive strengths. 
Valid national and/or local alternatives are 
generally also available. 

o Successful regionalism is highly context-specific 
and tends not to be general izable in a planned, 
orderly manner " . 

o While regionalism is relevant to coordination, it is 
not a panacea for coordination problems. Other 
non-reg 1 on a 1 al t ernat 1 ves are ava liable for 
coordination, and regionalism in fact adds to the 
amount of coordination required . 

o 8oth governmental and educational contexts present 
formidable constraints against regionalism. 

Observations of the regional educational laboratories suggest 
several pragmatic considerations. The prime responsibility for 
education nests with over 50 state-level education agencies and 
15,600 local districts. For the most part education agencies tend 
to be inwardly focused. When they look outward toward the nation 
as a whole, they are swamped by the numbers involved. Working 
together within regions makes the numbers manageable and still 
makes it possible for each agency to compare Itself with others 
having similar problems. In addition, travel and other 
efficiencies are involved. 

At another turning point in lab history, the NWREL re-examined 
the concept of regional 1sm and concluded that Its own regional 
definition was justified with reference to: 
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o Geography (including topography, population patterns, 
climate, trade and commerce, and communication 
patterns ) ; 

o "Kindred spirits" (feelings of natural kinship and 
professional ties) ; 

o Existing regional all iances ; and 

o Existing R&D performers (McClure 1977). 

Federal Specifications for Organization and Governance 

It 1s with the role of the labs 1n the regional organizational 

matrix that the RFP goes into greatest detail, spelling out the 

requirements in three tasks and two statements on the mission, 

functions, and governance structure as follows (NIE, 1984): 

o Task 1: Develop effective governance, management, 
planning , and eva 1 uat 1 on systems 

o Task 2: Work with and through existing organizations 
to Improve schoc" 1 s and cl ass rooms 

o Task 3: Work with state-level decisionmakers on school 
1 improvement i ssues 

o Statement 4: Laboratories serve designated regions. 

o Statement 5 . La bora tor i es have independent govern ing 
boards . 

A central notion was that labs must be both Independent of and 
neutral toward other regional organizati ;ns. There was recognition 
that education has diverse stakeholders and that the labs would 
have to be able to work with all of them without taking sides 1n 
any conflicts that might exist between other groups. 

In most major respects these specifications reflect the 
conf Igurational perspective. There was no vision of the labs 
playing specialized functional roles in a linear flow of knowledge 
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from producer to user. Rather, the labs were expected to work 
"with and through" other school improvement organizations, forming 
temporary systems to work on regional ly identified educational 
problems, and providing a variety of gap-filling R&D services. 
Laboratory Regions 

In the original program there were 19 laboratories served the 
contiguous forty-eight states, and one, the Center for Urban 
Education, focused on educational problems 1n major cities. 
Regions tended to be defined 1n ways that split many states between 
two or more laboratories, and, on average, there were only about 
two and one-half states per laboratory. states and SEAs were not 
viewed as key clients; preference was given to working directly 
with schools and school districts. 

Under the new program, only nine laboratories serve 50 states, 
Puerto R1co, the Virgin Islands, and the Pacific Island 
Territories. Regions are defined as groupings of whole states, and 
vary 1n size from four to nine jurisdictions. States and SEAs are 
viewed as key clients, along with other Improvement support 
agencies, while working directly with schools and school districts 
1s expected to be the exception rather than the rule. 

An overview of the characteristics of the nine regions 1s found 
1n Table 1. These are not necessarily the most Important or 
relevant regional Indicators, but are among those examined by 
program designers (adapted from data supplied by David Mack). 

It 1s apparent that there 1s considerable difference among the 
laboratory regions on these characteristics. For example, 
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enrollment varies from 2,055,000 in the NWREL region to 20,035,00 
1n the NCr-_»_ region. Population per square mile varies from 10 to 
383. It 1s noteworthy that NWREL stands either first or last on 



Table 1. Characteristics of Nine Laboratory Regions. 





No. 


General 


Pub/Pvt 


Land 


Pop 


No. 


No. 


Poverty 


Lab/ 


States 


Pop 


Enrol 1 


Area 


Per Sq 


Publ 1c 


Ed R&D 


5-17 Y 


Region 




(000*s) 


(000*s)(sq miles) 


M1 le 


School s 


Orgs 


(000's) 


AEL 


4 


15,548 


3, 150 


144,828 


107.4 


6,087 


125 


6C7 


FWL 


4 


28,647 


5,698 


461 , 763 


62.0 


8,877 


303 


778 


McREL 


7 


13,553 


2, 694 


573,462 


23.6 


8,572 


176 


343 


NCREL 


7 


48,660 


10,233 


379,331 


128.3 


20,035 


552 


1 ,272 


NE/I* 


9 


29,906 


6,818 


1 10,782 


270 .0 


9,441 


408 


948 


NWREL** 


7 


9,859 


2.055 


967,043 


10.2 


5 ,049 


114 


226 


RBS 


5 


24 ,680 


4,899 


64,421 


383. 1 


7,828 


291 


678 


SEDL 


5 


25 ,043 


5., 34 7 


549,203 


45.6 


10 , 746 


180 


1 ,072 


SEIL 


6 


30,609 


6,094 


289, 190 


105.8 


9 , 624 


271 


1 ,275 


TOT/AVG 


54 


226,505 


46,988 3, 


540,023 


64 .0 


86,259 


2,420 


7, 199 



^Includes Puerto R1co and the Virgin Islands. 

**Includes Hawaii, Marianas, Caroline and Marshall Islands, and 

American Samoa. 
Source: OERI 



almost every characteristic. It has the largest area and smallest 
popukdon and consequently the lowest density. Its poverty level 
1s near the bottom of the range. On the other hand, RBS has the 
smallest area and highest population density, but 1s closer to the 
averages 1n other characteristics. The need for the "with and 
through" strategy 1s evident from the number of schools 1n each 
region, ranging from 5,049 to 20,035. For any laboratory to 
provide direct services to even the smallest number would be 
difficult to comprehend. 

Of course those 1nter-reg1on variabilities mask considerable 
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intra-region heterogeneity. A range more extreme than that from 
New York City to the Virgin Islands (NE/I), or from Los Angeles to 
a remote county in Nevada (FWL), is hard to imagine. So it is not 
homogeneity but logistics that justifies a regional approach to 
knowl 9dge-based school improvement. Major metropolitan areas of 
all regions have more in common with each other than with rural 
areas of their own region, and vice versa. It is for this reason 
that both rural and urban themes assume importance for 
cross-laboratory coordination under Task 5. 
Special Considerat i ons 

In comparing the nine laboratories, several special 
considerations should be kept in mind. 

New vs . Old Labs . Six of the nine laboratories had been 1n 
existence for 19 years..when the current contracts began. Although 
they had to make adjustments by adding or subtracting states in 
thei * region or changing governance structure and revising 
programs, they were able to "hit the ground running." SEIL was a 
wholly new organization, although it was able to build on the 
experience of the pre-existing Southeastern Council for Educational 
Improvement . 

NE/I had to be organized as a new governing entity. While 
operations were sub-contracted to an experienced school improvement 
organization, The Network, its decentralized structure ,iad to be 
created and a new program put in place. In addition, its contract 
year operates three months ahead of other labs. Including Puerto 
Rico and the Virgin Islands in the region added considerably to the 
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heterogeneity of the region. 

NWREL was an experienced lab, but the decision to combine the 
Northwest and Pacific regions made the task more complex. 

Finally, NCREL was a totally new organization, but had been 
funded one year 6ar 1 1er than the others and so was actually In Its 
second and third years of operation 1n the two years being examined 
1n this study. In addition, NCREL was originally funded through a 
grant rather than a contract and was not subject to the same 
reporting requ1reme r .' s . 

In sum, we would expect that NE/I and SEIL would be just 
emerging from their developmental stage at the end of FY 1987, 
while NCREL would be further advanced Into program operations. 

Changes in Region . Among the experienced labs there were 
numerous changes, adding cr subtracting states. We have taken the 
view that all labs are continually having to negotiate their role 
1n relat " -)n to other regional o. ganlzat Ions . This Is magnified 
when one or more additional states are brought Into their orbit. 
So we i,;ight expect that within lab regions there would be variation 
among the several states 1n the speed with which a lab could become 
fully operational. 

Governing Boards 
In designing the new laboratory program the government was not 
content to fund laboratory-1 1ke activities in a variety of 
organizational contexts ; 1 1 requl red the establ i shment of unique 
regional organizations. They had to be independent and not simply 
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a subsidiary of some larger organization, such as a university, and 
still have strong ties to other regional educational Improvement 
organizations . 

The board of d 1 rectors was vl ewed not just as t he 1 eg a 1 
mechanism to control the organ 1z at 1 on but also to provide 1 Inks 
between the lab and Its major constituencies. This has both 
structural and functional aspects. (The analysis 1n this section 
1s based largely on the Governance and Organizational Status 
reports of 1986 and related sections of the proposals for 1986-90 
and 1988-90.) 
Structure 

There 1s a major continuing problem with the regional basis of 
laboratories. In our system of government, regional Institutions 
have an Inherently ad hoc character. No "laying on of hands" by 
OERI can give labs their legitimacy, although support and funding 
by the Federal government are Important. Legitimacy for their 
mission must be bestowed by other regional institutions. For this 
reason the governance structure of each institution is crucial. 
The key role of chief state school officers (CSSOs) in legitimating 
labs was recognized by requiring that all chiefs be offered the 
opportunity to sit or be represented on the board of directors. It 
was further required that the board reflect "a balanced 
representation of the states 1n the region, as well as the 
Interests and concerns of regional constituencies" (RFP Post-Award 
Requirements) . 

All the chiefs have accepted invitarlons to sit on the boards 

ERLC ^ 



CHAPT II GOVORG 



Page 11-13 



or send their representatives. McREL has the added stipulation 
that only the chiefs can vote; their representatives can 
participate 1n discussions but not vote. In most cases the chief 
1s listed as the board member, but 1n at least one case (SEDL) the 



serve also as the primary SEA liaison. 

All labs met the requirement of having equal representation 
from the states 1n their region (except that FWL treats California 
as two Sates, Northern and Southern California). 

Two of the labs also serve Island territories of the United 
States. NE/I serves Puerto R1co and the American Virgin Islands. 
Their chief education officers are "permanent" members of the 
board, and lab bylaws require that the islands, like the states, 
must have at least two members on the 41 -person board. 

NIE originally Identified a tenth region consisting of Hawaii, 
American Samoa, Guam, Northern Marianas Islands, and the Trust 
Territory of the Pacific Islands. However, for the first contract 
period, this region was combined with the Northwest. The NWREL 
Board serves as the board of both regions, but the needs of the 
Pacific Region are accommodated by the establishment of the Pacific 
Region Pol 1cy Board , composed of CSSOs and other educational 
leaders. In addition, an Indian Policy Board represents the 
Interests of Indian populations 1n the region. The relationship of 
these boards to the NWREL Board 1s not explained, and their 
activities are not described beyond such generalities as that they 
meet quarterly and "review progress , provide planning Input , and 
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establish priority activities" ( Plans for Years 3-5 , p. 6). 

Apart from t he CSSOs , t he boards must have about equa 1 
representation of educators and public participants. There is 
considerable variation in how they are selected. In most cases 
both educator and non-educator slots are filled t hrough a process 
and nomination and election by the board itself. However, some 
labs have a process whereby state advisory committees nominate 
(SEIL), and in some cases elect (NCREL) , board members. On the 
public side, SEIL allots three positions on a rotating basis to the 
chief executive officers of the state systems of higher education. 

AEL 1s unusual 1n allotting positions to the "designated 
representatives of state education associations (NEA affiliates), 
school administrator associations (AASA affiliates). and state 
associations of colleges for teacher education (AACTE affiliates)". 
[AEL Technical Proposal 1985, 1v] . 

SEIL allots three pos1 1 1ons on a rot at 1ng bas is to t he chief 
executive officers of state systems of higher education. 

FWL was established by eight public education agencies who 
signed a "Joint Powers Agreement" (amended in 1986 to add a ninth, 
the Arizona state Board of Education). Twenty-two members of the 
board are appointed by the nine signatory agencies , while seven are 
appointed by the Board Itself. In addition, there are three ex 
officio members: the Director of FWL, the Director of Regional 
Programs, and the Director of the Southern Service Center. 

Publ 1c members of lab boards are drawn "from "local school board 
members, parents, representatives of business and Industry, state 

ERIC Ji 



CHAPT II GOVORG 



Page 11-15 



legislators, state board of education members, and the community at 
large" (NIE 1985, 15). Thus, the "public" portion of the 
membership tends to include many Individuals with formal ties to 
education. For example, SEIL provides that the 12 public members 
will be drawn from the following "Client Service Groups" on a 
systematic rotation (SEIL Bylaws , Article III, Section 3). 
o Bus1ness/1ndusr *y representative 

o Intermediate service organization representative 
o Local school board member 
o Local superintendent 
o Parent 

o Private school official 
o Principal 

o State board of education member 
o State legislator 
o Teacher 

It was not possible to make a cross-lab tabulation of board 
membership because labs use different categories 1n reporting and 
because Individual members often can be classified in more than one 
category. Boards of directors tend to be large, ranging 1n size 
from 20 to 41. The log 1st ics of pre par ing briefing material s and 
bringing members together up to four times a year can be 
formidable. (One lab, SEIL, reduces some of these costs by having 
only annual meetings of Its 24-person board and making the 
six-person executive committee the primary governing body.) 

Laboratories are required to have independent governing boards. 
The meaning of this requirement seems to vary among the labs . In 
some cases (NCREL, SEIL), the lab and the funded organization are 
coterminous. However, laboratories are allowed to seek funding 
from other sources so long as additional work is consistent with 
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the lab mission. Some labs, like FWL, NWREL, and RBS, are 
considerably larger than the lab contract, and the OERI funded work 
1s a sub-set of the organization T s work. Thus , technical ly , the 
lab 1n the Western Region 1s the Western Regional Laboratory 
(WREL), also Identified as Regional Programs. Immediate oversight 
for OERI supported work 1s provided by the Regional Programs Policy 
Steering Commltt ee of the Boa rd. Similarly, RBS operates the Mid 
Atlantic Laboratory ar a subunlt. In the Northeast the winning 
bidder was a new consortium (as with NCREL) , but the lab was to be 
operated by one of the members, The Network, an existing service 
Improvement organization that had other sources of support. In 
this case The Network was not allowed to run the lab as a sub-unit; 
rather, a new Board of Overseers was formed for the lab, which then 
contracted with The Network to perform lab functions. Note that 
OERI practice 1s being followed 1n referring to the labs. The 
principle seems to be to use the name associated with the cognizant 
board of directors. Thus, for example, the Far West Board is 
recognized as the laboratory board, but The Network board 1s not; 
there 1s a separate "Board of Overseers" for NE/I. The corallary 
1s that 1t is all right for the work done under the OERI contract 
to be p3rt of a larger organization 1f that larger organization 
Identifies Itself as the regional educational laboratory and all 
work done by the larger organization 1s reasonably consistent with 
the lab mission. 
Functions 

Laboratory boards have the usual functi onal responsibi Titles of 
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corporate boards, Including establishing policy and monitoring the 
performance of management . Al 1 operate under bylaws and have 
establ ished Impressive arrays of policy manuals that cover topics 
such as personnel , contract administration, travel , and property 
administration; but they are of little interest for present 
purposes . 

Two other sets of functions, however, deserve mention. While 
planning and evaluation are regular functions of corporate boards, 
they are particularly crucial functions for laboratories, and 
boards ^re expected to play multiple roles at different stages of 
the complex planning process. This will be discussed 1n more 
detail 1n the section on planning. 

In addition, lab board members are expected to play a role in 
taking lab messages back to their constituencies and giving the lab 
visibility. As an example, here 1s how NE/I phrases these 
responsibilities ( Governance and Organizational Status Report , Vol . 
X> no page number): 



CONSTITUENCY BUILDING AND COMMUNICATION 

o Approve eligible schools, school districts, agencies, 
and organizations for affiliation with the Laboratory* 

o Build and strengthen relationships with Laboratory 
constituents, clients, and the general public 

o Serve as a channel of communication from the field to 
the Laboratory 

o Create Laboratory visibility 



*Th1s provision was later deleted. The lab still has affiliates, 
but there 1s no formal approval process (letter from Glen Harvey, 



12/6/88). 
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While not all labs are this explicit about these functions, It 
seems pretty clear that these expectations generally obtain for 
members of all lab boards. Here again, we see that the laboratory 
board 1s expected to assist the laboratory In negotiating a niche 
in the regional educational Infrastructure. 



Organization and Staffing 

The problem of how to organize a regional educational 
laboratory 1s complex. It Includes several Issues: program 
structure, organizational structure, the part/whole problem, 
organizational models, partnerships and networks, and methods of 
augmenting staff. 
Program Structure 

The structure of programs, projects, and activities derives 
from three major factors: (a) the task structure mandated by the 
government, (b) the priority areas chosen as a result of the needs 
and capabilities assessment process, and (c) educational 
Improvement and linkage strategies appropriate to the problems and 
contexts . 

The task structure required for government accountability 
obviously has to be accommodated, but 1t presents Its own problems. 
Only AEL and RBS use a simple classification of programs by task; 
All others use some form of a matrix design for at least some tasks 
and program structures. 

Task 1, relating to governance, organization, planning, and 
evaluation, is relatively straightforward. Virtually all labs 
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assign responsibility to the Office of the Director. If they have 
a special staff unit for planning and evaluation, that unit has a 
major responsibility for parts of the task dealing with needs 
sensing, capability assessment, self-assessment, and planning. In 
a few cases specific assignments are made to line units. 

Similarly, Task 3, work with state-level decisionmaker* gp 
school Improvement Issues, tends to be a self-contained program or 
project, although NE/I parcels out six Task 3 activities among four 
programs and SEDL distributes 10 activities among four programmatic 
themes . 

Initially, Task 5, collaborative activities, was a placeholder 
in that the specific projects had to be planned after the award. 
For most themes 1t took two years to develop plans. Plans for 
years three through five describe 11 collaborative projects, but 
not all labs participate in all projects. Where there is a 
correspondence or fit between the collaborative theme and another 
program or project, such as evaluation or state policy 
collaboration, most labs assign the theme to that program. 

The major problem comes in dealing with Tasks 2 and 4, which 
represent the core of the program and were expected to absorb 
between 60 and 80 percent of program effort. Task 2, "with and 
through," 1s a mode of operation, not a task 1n the traditional 
sense, although it tends to be interpreted as the 
d1ssem1nat1on/util izatlon function. On the other hand. Task 4, 
"work to create research and development based resources for school 
improvement" (emphasis added), is ambiguous and turns out to be an 
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umbrella that covers everything from applied research to collection 
of promising practices. The RFP does allow the conduct of applied 
research or action research. 

McREL Initially adopted a very narrow interpretation of thns 
task. No separate projects were Identified for Task 4; rather, the 
materials development activities of projects under Tasks 2 and 3 
were classified as Task 4. (In the plan for 1988-90 a new 
demonstration program 1n local sites has been formulated - a 
different Interpretation of Task 4, and of the "with and through" 
mandate. ) 

Two of the labs (AEL and RBS) appear to have self-contained 
program units organized by task (although RBS may have a modest 
amount of matrlxlng). But most labs use some matrlxlng across 
Tasks 2 & 4 1n which programs and projects are distributed across 
these two (and sometimes" other) tasks. Generlcally, such programs 
look Hke this: 



Task 2 



Task 4 



Program A 
Project 1 



Activity 1 . 1 
Activity 1 . 2 
Activity 2 . 1 



Activity 1 . 3 



Project 2 



Activity 2.2 
Activity 2.3 



Program B 
Project 3 
Project 4 



Activity 3 . 1 



Activity 3.2 
Activity 4 . 1 



Figure 2^ Generic M atrlxlng of Tasks and Programs . 



Organizational Structure 

Whether such a program structure represents ths way the lab 
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operates or simply an accounting device depends on the 
organizational structure chosen. If the organizational units 
coincide with programs and most work on the program and projects is 
done within that unit, then a program matrix such as that in Figure 
1 1s Just a way of being able to report to OERI 1n terms of the 
task structure. If the lab 1s organized by tasks or functions, 
program work is likely to be distributed across units. Virtually 
all labs have some functionally specialized units for Information 
dissemination that provide support to program units. 

While seven of the nine labs use some degree of matrix 
management, SEDL 1s strongest 1n Its avowal of this approach. 
Generlcally, Its structure for organizational units, tasks, and 
themes Ic.ks like Figure 2. In this organizational design. 



Themes 
A 



6 



ORGANIZATIONAL UNITS (GROUPS) 
I II III 

School Improvement School I mprovement LMC& 
Services Group Resources Group OCD* 



Task 2 



Task 3 



Task 4 



e 



Task 5 



e 



Figure 3 ; Matrix Organization of SEDL 
(Adapted from Proposal . p. 56). 

*Lab Management Council and Office of Communication and 
Devel opment . 
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programmatic themes are further divided into subthemes and 

activities. Tasks 2 and 3 are assigned to the School Improvement 

Services Group, which Implements knowledge utilization efforts, 

while Task 4 1s assigned to the School Improvement Resources Group, 

the knowledge production unit. Task 5 is assigned to the 

Laboratory Management Council and the Office of Communications and 

Development. Thus, the organizational units correspond to the 

tasks , but programs are matr ixec! across functional ly special 1zed 

units. The main axes of coordination seem to be for programmatic 

themes, 1n. charge of program coordinators. 

It may be helpful to provide a short description of each lab, 

combining aspects of program and organizational structure. The 

11st 1s roughiy 1n order of complexity, starting with the most 

simple arrangement . 

NCREL. A sma-1 1 laboratory with a small staff. 
Functionally integrated and thematlcally designed program 
units operate prSjects. Task activities are Identified 
only for OERI accounting. 

AJEL. Intermediate size. Six programs are grouped Into 
three tasks and appear to have self-contained staffs. An 
attempt to obtain the cont r i buted services of ret 1 red 
professors was abandoned. 

SEIL. A small laboratory with a small staff. Ten 
thematlcal ly defined projects operate under four 
programs, but neither programs nor tasks seem to be 
significant management categories . Projects are staffed 
by teams of individuals with multiple assignments and 
roles. There 1s reliance on positions contributed by 
SEAs and state systems of higher education. Also several 
projects are contracted out with the use of RFPs. 
RBS. A large laboratory with three non-lab programs. 
Six OERI programs 1n six functional ly defined units 
correspond to five tasks (two programs in Task 4). 

McREL. Small organization. Like SEIL, projects are 
staffed by individual s with multiple assignments and 
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roles. Lab 1s nominally organized by task, and there are 
task coordinators. Work 1s matrlxed across "strategies" 
or functions. Task 4 1s subsumed under Tasks 2, 3, and 
5. 

NWREL . A large organization with a significant amount 
of non OERI work. Heads of f unct 1onal ly defined programs 
supervise projects centers, or services. Six programs 
(partly functional, partly thematic) are matrlxed across 
organizational units and tasks. Tasks appear to be 
mainly OERI accounting categories. 

NE/I . Intermediate -size with decentralized structure. 
Six thematic programs operated under Field Services for 
Tasks 2, 3, and 4, with team staffs drawn from both 
headquarters and assistance ce~ er units. Executive 
Director, Associate Director, and Director of Planning 
and Communications have pieces of Tasks 1 and 3. (This 
1s descriptive of the lab for the period covered by this 
report. Subsequently, 1n January, 1988, the lab dropped 
the term "assistance center", consolidated program 
functions 1n the central office, and focused field staff 
efforts on field activities.) 

SEDL . Intermediate size. Two functionally defined 
groups. School Improvement Services (Tasks 2 and 3) and 
School Improvement Resources (Task 4) plus the Office of 
the Director (Tasks 1 and 5) matrix work across four 
programmatic themes , subdivided Into subt hemes and 
activities. Accountability to OERI Is by task, but 
organizational goals are stated 1n ten.is of themes. 

FWL/WREL . Large organization 1n wilch the Western 
Regional Educational Laboratory (WREL) is a unit of FWL. 
The Field Services Director is responsible for Tasks 2 
and 4, which are matrlxed across three thematic programs 
plus functionally specific units as necessary. Tasks 1 
and 3 parcelled out among staff units associated with the 
WREL management group. Has task coordinators 1n addition 
to line supervisors. Partial decentralization through 
establishment of the Southern Service Center at CSU 
Northrldge and placement of field agents 1n state 
capitals to serve the- Task 3 Policy Support Services 
Program. 



Looking across the labs, 1t 1s interesting that degree of 
complexity is not closely associated with size. No doubt some of 
the complexity could be reduced 1f OERI defined tasks in a manner 
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more compatible with program organization. 
The Part/Whole Problem 

Laboratories are allowed to take on work In addition to that 
funded by the lab contract, and 1n some cases the total 
organization 1s considerably larger than the tab. This raises the 
question of how the lab 1s distinguished from the larger 
organization 1n the organizational plan . 

In some cases the lab 1s a sub-unit of the organization. Thus, 
the Western REL 1s coterminous with the Regional Program of FWL and 
1s headed by Its own director, who reports to the Director of FWL. 
On the other hand, RBS uses a much more Informal mechanism: of 
their nine programs, six make up the lab and the other three are 
outside of it. All program heads report directly to the RBS 
Executive Director. 

McREL 1s somewhere in between . The execut 1 ve di rector of t he 
lab wears a seu^nd hat as principal investigator undar the lab 
contract. He then delegates responsibility for lab tasks 2-5 to 
the Coordinator of Lab Programs while retaining responsibility for 
Task 1. (It is not clear how non-lab work 1s administered.) 

The relationship between NE/I and The Network is complex. The 
lab proposal was submitted by a consortium of regional 
organizations, for which The Network was the fiscal agent. As a 
result of negotiations with OERI, NE/I was established as a 
separate legal entity with Its own board, and the conduct of work 
1s subcontracted to The Network. For its part, NE/I operates the 
laboratory as a separate organizational unit, but there is some 
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sharing of personnel with other units that are funded separately. 

Other labs do not say specifically how they handle the problem; 
presumably non-lab programs and projects are identified as separate 
work units below the office of the director. 

Both here and 1n the discussion of the Independence of governin 
boards above we observe one of those continuing issues that defies 
resolution. So long as lab can seek additional sources of funding 
(which may be necessary for their viability) the work done under 
the OERI contract 1s part of some larger whole and cannot be 
total ly independent . 
Organizational Models 

Several labs based their- organizational design on a th ughtful 
examination of recent organization theory and research, notably 
McREL , FWL and NE/I. 

Citing the theory of William Ouchi , McREL opted for the "M" 
model rather than the U-Form or H-Form. The "M H model is a 
variation of matrix management that creates independent units 
responsible for task specific activities and products but uses 
common service units. The maintenance of a business office and 
field staff 1n Kansas City seems to be for logistic- and historical 
reasons . 

FWL had a history of being more like a research center than a 
regonal laboratory. Not only were they more heavily weighted 
toward knowledge production than knowledge utilization, they also 
operated with a high degree of principal Investigator autonomy and 
a m1n i mum of pr ogr ammat 1c control. In planning for operation under 
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the new laboratory program they recognized the need to rethink 
these matters . They have st rived for an "organic" form of 
organization highly adaptable yet integrated 1n its functions and 
activities. The organizational design Incorporates a limited 
amount of decent ra 1 1zat1 on . A forma 1 relationship was establ 1s hed 
with the California State Ur 1vers1ty (CSU), 1n part to create a 
presence 1n Southern California where 1t had not previously 
operated. The FWL Southern Service Center was established on the 
campus of CSU Northrldge. In addition, Task 3 places field agents 
1n the state capltols of Arizona, Nevada, and Utah. 

NE/I was unique 1n Its Initial degree of decentralization. In 
the 1960s, when the Infrastructure of educational Improvement was 
far more primitive, a number of labs had tried decentralized 
structures. These regional offices were soon discontinued because 
1t was too difficult to keep field agents Involved in and up to 
date with lab activity (Salmon-Cox, 1980). NE/I felt that, 1n a 
new era when the role of the lab 1n working with other SIOs was 
paramount, a new attempt at decentralization was required. Their 
10 assistance centers were not 1 reestandlng , howe/er; they were 
housed 1n school Improvement organ 1zat ions that were part of the 
consortium sponsoring the lab proposal. Some difficulties were 
encountered with this approach. It had to be fine-tuned during the 
first eighteen months, and was abandoned 1n the second quarter of 
the third year. More specifically, some field staff were retained 
for field functions, and the term "assistance center" 1s no longer 
used for field units. What was terminated was the attempt to 
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involve field staff 1n program teams with central office staff. 
Partnerships and Networks 

Perhaps there was a day when it was clear where one 
organiztion stopped and the next began, but no longer. In this 
day of partnerships and networks, ties between organizations seem 
to have infinite gradations. We have already seen, in the section 
on governance r how other organizations or organizational sectors 
are formally represented on the lab boards of directors. In 
chapters below we will see how adjunct structures such as state 
advisory committees rjre created for needs assessment, evaluation, 
or dissemination, and will examine how the "with and through" task 
is implemented with partnerships, networks and other arrangements. 

One example will be presented here to demonstrate how deeply a 

lab can be embedded in its organizational context. AEL has very 

close ties to four other organizational sets : the CSSOs , state 

teachers associations, state admin istrators associations, and state 

teacher education associations. These ties operate on at least 

four levels. Each organizational set: 

o Has a state representative on the board of directors 

o Has one or more lab programs dedicated to working on 
its prcbl ems 

o Provides advisory oversight for the dedicated 
program(s) through an advisory committee of the board 
made up of the set representatives on the board . 

o Provides staff for the dedicated program in the form of 
shared staff, contributed staff, or volunteers. 

To sum up, lab organizations are not stamped from the same mold 
They exhibit considerable variation both in internal structure and 
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ways of relating to other organizations. The common threads are 
the need for flexibility in staff assignments and the desire to 
establ ish structures that assist them in being responsive to 
environmental press. 
Staffing 

A number of questions concerning staffing might be of interest. 

Unfortunately, 1t 1s not feasible to address those concerning staff 

characteristics, such as whether they are drawn from the research 

or practice communities, from the behavioral science disciplines or 

education, expertise in R&D functions and/or education problem 

areas, etc. But organizationally the labs had several choices to 

make. All have a corps of full-time staff, although 1n many if not 

most cases there 1s some splitting of time between lab and non-lab 

work. Variation comes in the degree to which that core staff are 

supplemented by part-time or contributed staff, consultants, or 

sub-contractors. Here 1s a run-down of some patterns: 

o AEL: Has a core staff in a central location with no 
field offices. The program on Professional Preparation 
and Resources 1s run by four 20% faculty members in 
area un 1 vers 1 ties . Sel f -assessment is conducted part 1 y 
through a subcontract for a third-party evaluation. An 
attempt was made to engage retirad personnel on a 
voluntary basis, but was abandoned. A program of 
Extern Grants provides low level support for doctoral 
dissertations and faculty research. 

o FWL: Has a core full-time staff 1n San Francisco and a 
"Southern Service Center" on the campus of the CSU at 
Northrldge. In addtlon, state assistance field agents 
( . 2 FTE each) are located in Nevada , Utah, and Arizona . 
In connection with Task 3 , a university f acu Ity member 
1n each of these three states serves as a State Policy 
Field Agent. Consultants are to supplement staff where 
specialized expertise 1s required for limited tasks. 
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o McREL: Has very small core staff 1n Colore 'o plus field 
office 1n Kansas City mostly for management support 
functions. Comm issions some papers and does some 
subcontracting 

o NCREL: Has very smaVi core staff 1n Elmhurst, IL. Uses 
contri butea staff, consultants, and subcontractors . 

o NE/I : Prior to its 1988 reorganization, had core staff 
1n Andover, MA, with some staff shared with other 
Network projects; other staff 1n 10 assistance centers 
(three in Andover and seven 1n field sites in host 
agencies ) . 

o NWREL: core staff operating out of headquarters in 
Portland; field office for Pacific Region 1n Honolulu. 

o RBS: large core staff plus considerable cost-sharing 
from partners in form of contributed staff 

o SEDL: central staff operates out of Austin TX. 

o SEIL: very small core staff; considerable use of 
subcontracts and consul t ants . 

Thus the labs exhibit considerable variation in staff size, 

reliance on consultants, contributed staff, and contractors, and 

placement of staff in one or multiple locations. 



Comments on Governance and Organization 
The concept of regionalism in education was examined. It 
appears that there is as much variation within regions as between 
them and that the primary need for a regional approach to the lab 
mission lies 1n logistical considerations. 

The nine regions served by the 1 a bora tor 1es vary cons iderably 
1n educational and demographic characteristics. While we continue 
to view the organizational Infrastructure of the several regions as 
loose and fluid, 1t seems clear that the labs have managed to 
establish important mechanisms that link them to other 

O 
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organizations at the governance 1 evel , chief 1 y through pol icies 
concerning membership on the board of directors and establishment 
of various advisory groups and other adjunct structures. The 
organization o* the labs at the regional level , outside the formal 
structure of the educational system, has its weaknesses . However , 
it 1s the very looseness of the educational configuration that 
seems to require an organization like the labs to play a catalytic 
role in bringing the disparate parts together in greater harmony. 

All labs group projects into programs. For a few the programs 
correspond to RFP tasks, but for most the programs are spread 
across the tasks. Most labs are quite ^sel^-conta 1ned regarding 
strsfr, although ell use consultants and sub-contracts from time to 
time. For a few, the use of outside resources is more than 
incidenta 1 . 

The task structure formulated 1n the RFP may be valuable for 
clarifying the mission of laboratories, but it does not seem to 
provide a useful way for labs to organize their programs or report 
to OERI . Whether some form of matrix management is needed to 
provide flexibility or merely provides a cross-walk for reporting 
to OERI by task is not clear. 
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CHAPTER III 

PLANNING* 
Introduction 

Federal Spec 1 f 1cat 1 ons 

An orientation toward rational systems 1s most evident 1n the 
required planning component. The RFP lays out three subtasks under 
Task 1 (NIE 1984): 

1.2 Assess regional needs, capabilities, and oppor- 
tunities and establ 1sh priorities for 1 a bora t or y 
activity. 

1.3 Prepare revised plans for future services. 

1.4 Conduct self-evaluation of laboratory projects and 
services . 

The text under these subtasks reveals a complex and sophisticated 
model of planning and management. Needs are to be assessed with 
such mechanisms as advisory committees, surveys, content analysis 
of media, and documentation of lab activities. Sel f -assessment 1s 
expected to serve a formative evaluation function and feedback Into 
the planning cycle. The board of directors should play multiple 
roles as both a source and a consumer of needs and sel f -assessment 
Information . 

In addlt 1on , each lab must conduct a census and analysis of 
R&D and service 1mpro /ement organ 1zat1ons 1n the region to assess 
the capacity of the Improvement Infrastructure 1n relation to the 



^Planning done under earlier planning grants is not part of this 
review except as 1t is reflected 1n the plans for 1986-90. 
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the needs Identified as well as to its own capacity. Analysis of 
these data leads to setting priorities and planning specific 



useful for both f1ne-tun1ng day-to-day operations and making major 
program planning decisions, such as developing the three-year plan 
for 1988-90. 

Each lab was asked to provide a graphic displ ay of the total 
process. To illustrate, Figure 5 shows the process described by 
FWL/WREL, which is typical. Shown as a systems chart, it indicates 
the highly rational and complex set of tasks required. 
Planning Models 

The process described above seems to represent the convergence 
of thinking from several conceptual starting points. "Evaluation , " 
"needs assessment , " and "planning" have all been used as the master 
concept subsuming all stages of the process. For example, under 
the evaluation rubric, Stuff lebeam et al (1971) developed the CIPP 
model that encompassed these processes under the headings of: 



Beginning with a system approach to planning, Kaufman (1979) 
elaborated six types of needs assessment , each associated with one 
of the six steps of a general systems problem-solving model: 



programs and services. 



This process 1s viewed as continuous, and 



Context evaluat 1 on 
Xnput evaluation 



Process evaluat i on 
Product evaluat 1 on 



Al pha : 



I dent if y probl em based upon need 



Beta : 



Determine solution requi rements and i dent if y 
solution alternatives 
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Figure 4. The Far West Labcratory Planning Process 
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Gamma ; 



Sel ect sol ut 1on st rat eg ies from among 
alternatives 



Delta : 



Imp! ement 



bps 1 1 on : Determine performance effectiveness 



Zeta : 



Revi se as requi red 



Similarly, working from a chain model of educational needs , 
Waks (1979) identifies: 
Context Inquiries 
At cainment needs 



Process neeas assessment 

Maintenance needs assessments 

Resource needs assessments 
o Personnel assessments 
o Organizational assessments 
o Fad 1 1t 1es assessments 
o Financial assessments 



Finally, a field of strategic planning has been elaborated that 
encompasses external environmental scanning. Internal capacity 
analysis , participation cind Involvement , mission and strategic 
goals, implementation/linkage plans, and monitoring and assessment 
(Stelner 1979; McCune, no date). Several labs have explicitly used 
variations of strategic planning (McREL, SEIL). 

All t he^e model s cover more or less the same set of processes 
but use a different master concept, depending on the starting 
point. For our purposes 1t 1s only necessary to note that the 
writers of the lab RFP specified a set of requirements for Task 1 
that was consistent with these different traditions. 

From an examination of these models and the subtasks required 
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of the labs, the following general planning model can be inferred: 

Reg i ona 1 Needs 



Regi onal ^ } Laboratory 



^ Laboratory Program 



Capacity w Capacity 

Figure 5 . Strategic Planning Model . 

According to this model , the lab first assesses the region's 
needs and the capacity of Its Infrastructure to meet them. It then 
derives Its program from this analysis and an assessment of its own 
capacities . 
Issues 

This highly rational planning process appears to be in tension 
with the previously noted recognition that labs must operate in a 
fluid configuration of educational agencies. Lindblom and Cohen 
have argued that social problem-solving is basically an interactive 
process and aoes not evolve 1n any simple way from professional 
social Inquiry (1979). Clark maintains that plans more often 
represent an understand 1ng of what has a 1 ready been done rather 
than a useful prescription for future action (1980). Whether the 
Imposition of rational planning methods on the labs represents a 
conflict and Inconsistency or whether the planning process 
constitutes a useful adaptive mechanism for operating in a 
non-rational environment is something I hope will emerge from the 
analysis. Other Issues concern the extent to which the general 
model has been Implemented, and the degree to which plans and 
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activities flow logically from the planning process. 
Organization of Chapter , 

The sections that follow discuss each of the major components, 
needs sensing, capacity assessment, and sel f -assessment 1n turn, 
and examine how the planning and management process all fits 
together . 

Needs Assessment 

Theoretical Basis . 

Needs sensing or assessment seems to correspond to context 
evaluation 1n CIPP model, contextual Inquiry in Waks' model, and 
environmental scanning in strategic planning. While the Idea of 
measuring educational needs appears intuitively simple and 
straightforward, Waks (1979) points out are at least four meanings 
of "need": 

o The norm-based sense of need: something required by a 
prescriptive rule or law, e.g. "one needs a license to 
go fishing here" 

o The goal-based sense of need: a necessary means to 
attainment of an explicit goal 

o The motive senses of need: a want or desire 

o The injury sense of need: when lack causes harm 

These senses are not mutually exclusive. The labs primarily 

seem to use a goal -based sense of need with some overtones of 

Injury or gap. In the planning context, needs assessments are 

practical ways for measuring needs and therefore goal s . 

Goals are an Important focus 1n most needs assessment 
studies. By determining needs, goals are being verified 
or validated, selected from alternatives, refined, or 
converted into more specific objectives. Needs 
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assessments are tools for determining the goals of a service 
agency (Waks 1979, p. 63). 

The sine qua non of a laboratory is its orientation toward 
meeting regional educational needs. The needs assessment function 
is thus the keystone activity, and in qui r ing about the sources and 
methods used in determining those needs is essential. 
Sources and Methods * 

As has been noted above, all labs rely on their boards of 
di rectors to some extent for needs sensing input . Staff contacts 
and information gleaned in meetings of all kinds are also 
important. In terms of more formal methods of data collection and 
analysis, tremendous variation exists among the labs.- Some rely on 
an analysis of one or two types of data, while others (FWL, NWREL) 
use the 1987 report to summarize a long history of detailed studies 
using a wide variety of data sources. 

Adjunct Structures . The f i rst method of needs assessment 
involves the formation of adjunct struct' res, sucS as advisory 
committees and formal partnerships. In general, there are two 
kinds of advisory committees: those organized by state and those 
organized on a program or project basis. The new labs in 
particular opted for elaborate advisory structures. Note the 
foil owing memberships : 



*This section is based largely on the needs and capabilities rep rt 
required as a contract deliverable as part of the continuation 
proposal for the third through fifth years. Other needs data are 
to be found in the original five-year proposal . 



o NCREL: 140 members in 7 State Advisory Councils 
o NE/I: 219 members in 9 Advisory Committees (later 
di ssol ved ) 
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o SEIL: 109 members in 6 State Advisory Committees 
Among the old labs, NWREL and SEDL convened ad hoc state 
advisory groups for the recompet i t ion , but have not continued to 
Involve them as such. (SEDL has involved some of the same 
1! dlvidvjals 1n other ad ( «^nct groups-) State advisory committees are 
partly needs-sensing mechanisms, partly dissemination/liaison 
vahicles, and in some cases have a role in the governance of the 
lab through nominating or electing board members. Committee 
discussions make it possible for representatives of different 
constituencies to share anc' compare their views and give the 
laboratory a more in-depth and qualitative view. 

It is interesting to speculate on why the new labs ace far more 
likely to rely on adjunct structures. Perhaps the old labs are 
sufficiently tier 1 ir, to the educational infrastructure of their 
regions through their governance and program activities, while the 
new labs, having new governance and program structures, need the 
additional mechanisms on at least a temporary basis. Note also 
that state advisory groups are cross-constituency groups; they 
attempt tc represent a cross -sect ion of educational stake hoi ders in 
the state. By contrast, the older labs appear to favor more formal 
ties to specific constituencies . W< have a 1 ready noted how four 
groups are represented directly on the Board of AEL, and how nine 
signatory agencies appoint 22 members of the FWL Board. 

For program purposes , RBS has establ ished formal partnerships 
with five groups (some new, some pre-existing): 
o Rural Exch ?,->ge ^etwon 

o Maryland Council of At_1sta.it Superintendents 
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o Intermediate Units of Pennsylvania 

o Mid -At 1 ant ic Met ropol i tan Counc 1 1 

o Urban School Superintendents of New Jersey 

While these groups serve primarily as vehicles for program 
operations, they are also sources of needs input. 

Program and project advisory groups (to be discussed more fully 
1n connection with Tasks 2-4) serve primarily as input for 
f1ne-tun1ng existing activities rather than planning new 
1n1 t1at1 ves . 

Quantitative Data. Labs employ a wide variety of methods for 
gathering quantitative data, including surveys, content analysis of 
media, secondary analysis of published economic, demographic, 
social, and educational indicators; and logs of program activity 
such as telephone contacts and services requests . The profile of 
methods used by each lab is shown in Table 2. 



Table 2. Methods Used to Collect Needs Assessment Data. 



Lab Meet ings Surveys Content Indicators Documen ta- 

Ana lysis tion 

AEL + o 

FWL + + + + + 

McREL + + + + 

.JCREL* + + - r + + 

NE/I + + f + + 

NWREL + + + 

RBS + + + + 

SEDL + + + 

SEIL + + o 



*NCREL is on a different schedule and no separate needs 
assessment report was required in 1987. Proposal for 
88-89 was used. 
KEY: + = method used and analysis provided 

o = data collected but no analysis presented 
- ■ no data collected 
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All labs used multiple methods, and some used all. This kind 
of profiling, however, 1s not very adequate because it masks the 
fact that a " +" in the table for one lab might mean a cursory 
discussion in a few paragraphs and for another it might mean a 
detailed analysis in a separately bound report. 

Surveys . All labs except SEIL conducted at least one needs 
sensing survey , and some 1 abs conducted two on three. Virtually 
all samples were purposive rather than random. In some cases a 
single group, e.g., superintendents, was studied, while in others 
many different groups were surveyed. Response rates varied from 
poor to mediocre. Data 1n survey format were also collected from 
members of advisory committees and participants 1n conferences, 
workshops, and other meetings. 

Generally, the procedure was to ask respondents to rank or rate 
Items from a list of possible educational problems. In at least 
three cases (AEL, NE/I, and SEDL), a common met hodol ogy developed 
by Grunlg was employed. This involves .skmg for each item whether 
the respondent (1) ever thinks about the Issue, (2) could make a 
difference, and (3) 1s connected to the Issue. Although a great 
deal is made of the fact that this approach is based on 
communicatlcns theory , that theory is not expl alned or used to 
provide any special Interpretation of the data. Indeed, although 
it seens possible that the three Items might form a scale, no 
attempt is made to examine the interrelations of the three 
responses . 

Beyond that, many objections have been raised to using attitude 
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surveys for measuring needs (Waks 1979). Which sense of "need" is 
being employed? Are they attainment needs, process needs or 
resource needs? How does one distinguish between needs and wants? 
How does one weight the responses of different stakeholders? 

Imp licit in the needs assessment approach is the i dea that 
needs can be prioritized and that need patterns will vary from 
region to region and state to state. Yet many of the issue 
statements are at the level of "improve the quality of teaching." 
But surely at this level, which is the level of the major 
components of the educational enterprise, every lab needs to have 
the capability of providing some KP or KU service. The need to set 
priorities comes only at lower levels of specificity where choices 
must be made among lower level objectives — e.g., which aspect of 
teaching quality is most in need of attention? Further, at some 
point the choices to be made become a technical issue that must be 
based on the best evidence from research and/or experience and not 
on attitudes or values. To summarize, if there is a place for a 
survey approach to needs assessment, it needs to find a place 
between the broad components of education on the one hand and the 
1 evel of technical design on the other . 

Indicators . All labs examine a variety of statistical 
Indicators to demonstrate their understanding of their regional 
context , Including economic, demographic , social , geographic , and 
educational data from many sources. 

Strategic planning is the model often used either explicitly 
(McREL) or Implicitly, as evidenced by the frequent reference to 

ERIC 5o 



CHAPT III PLANNING Page 1 1 1 -1 2 

"environmental scanning" as the type of analysis employed. The 
ana lyses presented vary from superficial to soph i sti cat ed. McREL 
makes a plausible case on the basis of analysis of major economic 
shifts 1n Its region for making the restructuring of education its 
broadest goal . 

Attainment need? 1n Waks T sense are identified through analysis 
of NAEP and state ter>t 1ng data . RBS emphasized this type of 
analysis and published an analysis of regional test results for 
1978-86. This type of analysis led to the Identification of At 
Risk Youth as a program in several labs and as a cross-lab 
coordination theme . 

Content Analysis of Trends. This category covers several 
different methodologies. Several labs, Including AEL , McREL , and 
NE/I, employed the methodology popularized by John Naisbltt 1n 
Megatrends (1982) in which newspapers , magaz 1nes , profess ional 
Journals, and newsletters are systematically collected, coded, and 
placed 1n a database. This category also encompasses the more 
traditional scholarly analysis of publications such as needs 
assessment reports of other regional or national organizations. A 
review of major national , educational reform reports is one type 
(SEIL) and documentation and analysis of state legislative reform 
activity 1s another . 

Documentat Ion of Operations . Many labs have devised systems of 
logs and reports for gathering data 1n the course of their normal 
operations that provide grist for the needs assessment mill. These 
Inc 1 ude 1 ogs of tel ephone contacts, records of requests for 
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assistance, telephone and mail followups of participants in 
conferences and workshops, etc. NE/I devised elaborate systems of 
this kind , perhaps prompted by its decent ra 1 ized structure; its 
Initial system proved to be too cumbersome and had to be modified. 

Comments on Needs Assessment . The above section has described 
the range of methods used by labs for needs assessment, but it has 
not conveyed the considerable range in the depth and quality of the 
analysis . 
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Regional Capabilities 
Federal Specifications . 

The RFP requirements for studying regional capabilities were 

lumped with some others in Subtask 1.2: 

Assess regional needs, capabilities and opportunities 
and establish priorities for laboratory activity (p. 
19) . 

In addition, it was noted that 

...the assessment process should include a census and 
analysis of research and development organizations and 
the service Improvement organizations in the region in 
terms of their school Improvement activity (p 19). 

It was explained that "Data on their activities are crucial to 

Identifying gaps in services and possible partners for 

collaboration with the laboratory" (p. 19). 

A further explication of the rationale behind this requirement 

1s provided by Clark and Carroll (1980): 

Understanding the region means much more than 
1dent 1f y 1ng regional " needs" however they are defined . 
It means having an inventory of KP, KU , and school 
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1 mprovGment act i v1 1 1es al ready underway - thel r 
strengths and weaknesses; an understanding of the 
incentives and disincentives which Influence agencies 
and actors 1n the KPU community; a feel for new or 
modified activities which are feasible and acceptable; a 
sense of the Intrinsical 1 y Interesting needs or problems 
1n which Individual agencies are wil 1 ing to invest their 
resources (p. 4-1). 

Nature of Data Available 

In st rat eg ic pi ann i ng a com par i son of env i r on men t a 1 press 
(demand) with Institutional capacities (supply) is an essential 
element of the planning process (Steiner, 1979). OERI sought to 
focus on this type of analysis by requiring a separately bound 
"needs and capability assessment report" as part of the 
continuation proposal for the third through fifth years. It also 
required a statement of the lab's own organizational capability as 
part of the technical ..proposal . 

The assessment of regional capabil i ties was the most poorly 
Implemented component of the planning model. Why this should be so 
1s not clear given the fact that a useful model for maintaining a 
database on Institutions performing RDD&E or policy studies had 
been provided a few years earlier. NIE had sponsored a universe 
survey by the Bureau of Social Science Research of these 
organizations. Its conceptual framework and methodology had been 
detailed in the final report, ARROE — American Registry of Research 
and Related O rganizations 1n Education . by Joanne Frankel , Laure 
Sharp and Albert Blderman (Frankel, Sharp and Biderman, 1979), 
available in ERIC. A derivative directory of organizations was 
published by FWL (Lehmlng, 1982). 
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There appears to be one source of conf usi on concerning t he 
purpose of a regional capabilities assessment. A distinction can 
be made between KPU capacity for I nternal purposes ( e.g. the 
evaluation unit of a city school system) and as a service for 
others (e.g. evaluation contractors 1n the private sector). Yet 
the RFP asked only for a census of service providers, and 
apparently most labs limited themselves to this perspective. Thus, 
for example, 1n publishing a directory of educational resource 
organizations, RBS eliminated SEAs and LEAs because they did not 
offer services to others. (But surely SEAs provide services to 
LEAs even \f they aren't "for hire" for that purpose). No doubt 
RBS f s services to the Philadelphia school district 1s limited by 
the fact that they have a considerable capability of their own. 
However, in negotiating its niche in the regional infrastructure, a 
lab must know whether 1t is dealing with SEAs and LEAs with KPU 
abilities. Perhaps the lab distinguishes between the information 
needed for Its own purposes and that which would be useful to 
others 1n a publ 1shed directory, but the analyses in the planning 
documents appear to be limited to service providers and Ignore 
or anizatlons with internal capacity only. 

Another purpose of the resource inventory concerns the lab 
commitment to a "with and through" strategy. Labs need to identify 
other organizations they can work with and the type of relationship 
that might be established. In this context, it is well to 
recognize that some of these organizations may be 1n competition 
with the lab. Pe-hs-s this is the reason that RBS did not include 
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for profit agencies in its directory. 

Only R8S has published a current directory, although at thi 
writing NE/I has one in press. FWL has been maintaining such a 
database for some years , but i ts most '-ecent di rectory was 
published in 1985. However, FWL has clearly gone the farthest over 
the years in careful ly examining regional capabi 1 ities . Indeed . 
their report shows a two-page bibliography of laboratory studies, 
other regional studies, and national projects that provide analyses 
of the educational KPU capabilities of all kinds of institutional 
performers and of interorganizat ional networks. FWL would be in a 
position to provide leadership to new labs in making this a more 
realistic aspect of their planning. 

Several other labs (McREL, NWREL, SEDL) achieved minimum 
compliance with this RFP requirement by providing lists of 
organizations by type, state, and contact person. But without 
information on areas such as field of expertise, type of K M 
service, primary clients, and personnel, such lists are of limited 
planning value. Possibly each has more data in its files than it 
includes in its listing, end no doubt staff have personal knowledge 
of many other regional organ izat ions . 

No lab provi des an ana 1 ys is model for rel at ing capabi 1 i ty and 
needs information- SEDL provides an "environmental analysis" based 
on the market research model of Philip Kotler that contains a 
discussion of market environments and competitive en vi ronnrients and 
that includes a discussion of intermediaries ("with and through' 
organizations) and agencies offering competing services, b"t it is 
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not clear what data they are working from. 

Other 1 abs sought to finesse this requi rement by including one 
or more Items in surveys of clients or constituents that asked 
about sources of help for different kinds of problems, or about 
opinions of the most appropriate roles for the regional laboratory. 
But this methodology may overlook some potential resources, and 
there are obvious difficulties 1n relying on second-hand 
Information . 

Comparison of Lab Regions 

So far we have been discussing the collection and use of new 
Information on regional KPU capacity. At the time the current lab 
programs were being planned, data on educational KPU organizational 
capacity were available by state and region (Frankel et al 1979). 
A crude comparison of the nine regions is shown in Table 3, which 
displays the percentage distribution of four kinds of KPU 
organization across regions, along with the distribution of 
population and OERI funds. 

Clearly the distribution of KPU organizations of all kinds 
closely follows the distribution of enrollment. There 1s mere 
variability 1n the profiles of specific kinds of organizations. 

The Interpretation of these data is complicated by another 
consideration. Some service needs are enrollment dependent and 
some are not. For example, if all the teachers of a certain type 
need a kind of retraining, then the size of the task depends on the 
number of teachers. But the amount of effort required to conduct a 
state policy analysis may not differ greatly 1n a small state and a 
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Table 3. Percentage Distribution of Educational KPU Organizational 
Resources, Pub! ic/Pr ivate Enrollment, and OERI Funds, by Lab/Region. 



Ed KPU Organizat 1ons8 FY 1 987 Pub/Pvt 

Lab Small Large OERI Enroll- 





Publ ic 


Public 


Pvt 


Academic 


Total 


Funds 


ment 


AEL 


5 


4 


4 


6 


5 


9 


7 


FWL 


19 


7 


13 


10 


12 


14 


12 


McREL 


5 


9 


8 


8 


7 


8 


6 


NCREL 


24 


18 


14 


26 


23 


8 


22 


NE/I 


14 


9 


26 


16 


17 


1 1 


15 


NWREL* 


6 


7 


3 


5 


5 


17 


4 


RBS 


9 


13 


22 


9 


12 


14 


10 


SEDL 


9 


13 


4 


7 


7 


9 


1 1 


SEIL 


9 


21 


6 


13 


1 1 


9 


13 


Total 


100% 


101% 


100% 


100% 


99% 


99% 


100% 


Number 


537 


149 


478 


1 ,268 


2,432 


$16,975K 


46.988K 



*Does not Include Pacific Territories 

# Educational KPU organizations typology: 

Small public: ISAs and LEAs with enrollments of 10,000-49,999 

Large public: SEAs and LEAs with enrollments of 50,000 or more 

Academic: Institutions of higher education 

Private: residual category 
Source of organization data: Frankel , Sharp, and Blderman, 1979, 

Appendix F. Detail may not add to total because of rounding. 



large one. 

The table also shows the distribution of 1987 OERI funds. We 
can begin to see here why defining an equitable distribution of 
these funds is so difficult. One might argue that funds should 
follow the distribution of population or enrollment. However, some 
allowance seems to be necessar/ for factors such as different 
numbers of states 1n a region and t he loglst ics probl ems found in 
regions with large land areas and dispersed populations. On the 
other hand, 1f the existence of other KPU organizational resources 
1s taken into account , some kind of compensatory f ormu 1 a seems 
appropriate, i.e., more funds should go to regions with smaller 
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regional capacity. Another consideration that came Into play was 
the fact seme labs are in a developmental stage while others are 
o per at i on a 1 . No precise f ormu la was ever worked out . The actual 
distribution apparently reflects some intuitive balancing of the 
above factors, plus one other: a hold harmless provision that 
enabled previously existing labs to receive no less than their 
previous al location (conversation with David Mack). 



Self -Assessment 
Federal Specifications . 

It is useful to approach t sel f -assessment function by 

reviewing some chronology. The RFP identified sel f -assessement 

(SA) as a sub-task of Task 1, placing it within the strategic 

planning cycle where five objectives were listed: 

o Assuring that work performed meets appropriate 
standards for quality and efficiency 

o Assuring that the 1 a bo rat ory carries out the 
m1ssion,funct1ons and purposes for laboratories 

o Using evaluation information as a basis for planning 
adjustments , f ol low-on activities , and new 
Initiatives 

o Making staff evaluations 

o Contributing to knowledge about effective strategies 
for Improving education through carefully designed 
studies of how its own dissemination and improvement 
efforts are working. 

is quite a diverse set of purposes! The first three 
cl ear 1 y bel ong in the pi anni ng eye 1 e . However , the use of 
make staff evaluations concerns a different management 
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function. And the last 1s not a management function at all, but 
rather is a research objective. If one goal ties all these 
purposes together it 1s that of using information to renew and 
Improve the functioning of the organization. 

An annual SA report was requl red . Apparent 1 y some confusion 
existed about what was to be Included 1n the report, and toward the 
end of the first year the contracting officer Issued guidel ines 
covering both the format and content of the report (Wormwood, 
1986). Broadly, the report was to include an abstract, the design 
of the sel f -assessment , results, and use. 

The reports submitted for the first contract year varied in 
content, quality, and detail. NCREL, which had been funded on a 
different cycle and operated under different requirements, did not 
submit a report. Somot labs simply listed all their activities and 
sub-activities and reported on whether each had been completed . 
Some were short sub-sections of the annual report. In other cases, 
they were longer, consisting mostly of a rehash of SA design, 
either as described 1n the proposal or subsequently modified. Some 
submi tted the raw data from client surveys , exit interviews from 
workshops, and the like, 1n mind-numbing detail. A few supplied 
reports that were reasonabl y thoughtf u 1 in coverage of at 1 east 
some of the suggested evaluation questions, but appear more 
judgmental than empirical. Under these circumstances, an anal ys is 
of the first year reports probably would not be very fruitful. 

It also became clear that, 1n addition to playing an internal 
role in each lab's planning and management system, SA was viewed as 
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part of the OERI effort to evaluate the Laboratory Program. A 
consultant, Jerry P. Walker, was engaged to visit every lab to 
examine Its evaluation-related activities and to elicit the views 
of lab personnel concerning OERI evaluation plans. It was during 
the second year that plans for the 1986-90 fiscal years had to be 
prepared, at which point the 1986 SAs were available, but those for 
1987 were not. Each lab was visited by a site review team as part 
of that process. 

No new quldelines were prepared for the 1987 SA report, but 1n 
December 1987 Charles Stalford, who was 1n charge of the OERI 
evaluation effort, confirmed that the Wormwood guidelines were 
still 1n effect and suggested some additional questions that might 
be addressed (Stalford, 1987). These emphasized the use made of SA 
by the governing board, an analysis of the strengths and 
limitations of the lab's capacity for SA, and Identification of the 
activities and products that the lab considered the most productive 
and why. Particular Interest was expressed in knowledge coming out 
of SA activities that might begin to yield generalizations about 
the strategies that worked best. 

OERI also prepared an Evaluation Plan for 1987-90 . which stated 
that the three components of the evaluation plan for regional 
educat 1ona 1 laboratories were self-assessment, performance 
evaluation, and program evaluation. Labs were urged to report 
their shortcomings and strengths candidly, trusting that "While 
OERI has access to such reports as "del 1 verables , ' the Agency 
understands their v in-process T nature and will therefore treat 
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information in them carefully" (OERI , 1987, p. 5). Further, the 
idea of col lec ting uniform types of SA data across labs was 
disavowed. Although no additional specific guidelines were 
provided for SA , the following six questions were listed as guiding 
the overall evaluation, and were cited in some of the 1987 SA 
reports : 

o What is the evidence that the laboratory has had a 
positive effect on educat ion a 1 practice within its 
region? 

o Are 1 a bora tor y products and activities wel 1 received 
by educators and stat e-1 evel dec is ion makers? 

o Does the laboratory have productive working partner- 
ships with organizations in its re< ion whose 
cooperation is required to accomplish mutual 
ob ject i ves? 

o Are the laboratory costs cons' tent with those of 
other labs for similar types of activities? 

o Are constituent individuals and organizations in the 
region aware of Vie laboratory, its program and its 
products? 

o Are t he laboratory r s products aid act i vi t i es of high 
qual i ty? 

While there is considerable overlap with other statements of 
evaluation objectives , there are a cr'iple of new ones. The focus 
on lab visi bi 1 i ty is new (but probab / a very weak indicator ) . The 
focus on cost efficiency nas been adapted to an intra-lab context 
by some labs. The most interesting addition is the f ocu^ on 
establishing "productive working partnerships/' which seems to be 
an important indicator of success with the "with and through" 
strategy. The pres i descriptive synthesis report is one of three 
components of progr evaluation in the OERI plan. 
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The analysis in this section is based largely on the 1937 
self-assessment reports. (Again, NCREL , which was funded out of 
cycle , was not required to submit this report . It did provide some 
self-assessment information in its plan for 1988-89.) Designs that 
were described .in the proposals often proved to be too ambitious to 
accomplish, and as a consequence 1987 sel f -assessment efforts often 
contain revved designs, diminished results, or both. Some 
contract compliance data come from annual reports. There is a 
tendency to duplicate Informati on in the annual reports, needs 
assessment reports, and self-assessment reports, and OEPI would do 
well to sort out what kinds of information should go into each. 
Problems of Analysis 

We have seen that the objectives for SA are diverse. In its 
broadest sense, SA encompasses the entire management system of the 
lab, including the collection of all information required for 
derlsl on making , the a 1 1 oca t i on of res pons 1 bi 1 i t ies among posi t ions 
and units for collection and review of information, the substance 
of the information collected, and the resultant actions taken. The 
technology requ'i red just for inf or met 1on col 1 ec ted is so extensive 
and complex it s almost impossible ':o summarize. For example, AEL 
lists 18 sources of evaluation information and 29 forms of 
documentation for 12 "generic" activities or products. Although 
not all kinds of information are coll ec ted f u ^ every activity or 
product, there are 191 information/product combinations . RBS 
identified 15 kinds of products and ^3 kinds of activities to be 
eva 1 uated . While other 1 abs have not provi ded as complete a 
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documentation of what Is involved, this kind of detail is probably 
not atypica" . 

The use of much of this information is embedded in day to day 
lab operat ions. It 1s the stuff of weekl y staff meet ings at 
project, program, and Institutional levels. Seme of this gets 
aggregated to quarterly reports , annual reports, and SA reports 
(sometimes in too much detail), but much does not. Summarizing 1t 
all woul d serve no useful purpose . The cha 11 enge becomes one of 
sorting out the wheat from the chaff . Some labs seem hard pressed 
to find a middle ground between presenting raw, uninterpreted data, 
on the one hand, or presenting judgments and assessments, often of 
some Interest, but without underlying evidence on the other. 

For prasent purposes we have the problem of differences between 
designs and reports. Some of the SA designs, usually found In the 
original proposals but sometimes revised In later documents, appear 
comprehensive and sophisticated; but they are not always feasible, 
and both n€iw and old labs have had to rethink and revise. And 
always there is the nagging question whether overemphasis on SA may 
not amount to stopping the train to blow the whistle. One way of 
coping with this problem has been to take the position that not all 
aspects of a comprehensive design need be impl emented every year ; 
rather , each year 1 s SA report shoul d focus on a few appropriate 
components . Us ing this rationale, most 1 abs have chosen to delay 
focus ing on the difficult task of measuring impact until the last 
year or two of the current fivs-year contract. 

Another probl em is that of distinguishing between ad hoc 
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information and general knowledge. Both are needed, but it is 
important to dist inguish between them . For example , data on 
whether lab constituents are aware of and value the lab is ad hoc 
Information, but it is useful for formative evaluation purposes and 
in determining whether the l^b is meeting certain accountability 
objectives. Data on whether a particular lab strategy is likely to 
lead to lab visibility and acceptance may be the beginning of 
building a knowledge base on which strategies are more effective 
under given conditions. Every lab is expected to spend some 
portion of its SA energy in contributing to gt^eral knowledge of 
this kind (see RFP) . 

Self -Assessment Designs and Mechanisms 

The analysis cf SA is composed of lab profiles on the evaluation 
questions ad*' sed r ..the assessment models or images invoked, and 
the mechanis. _ used to Implement the design. The data are found in 
Tables 4 and 5. While ideally a tabulation would have been made 
linking typ^s of data collection and other mechanisms with 
evalua^'on questions, the necessary data were not always available. 
However, examples and impressionistic statements about these 
connect ions appear bel ow . 

A review of potent ic 1 eval uat ion quest ions in the various 
sources mentioned in the "Federal Specifications" section above and 
from the lab designs yields a considerable list. The several 
sources are not consistent , and there is no impl icat ion that all 
are applicable. 

It is useful to distinguish two kinds of evaluation questions. 
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Design 8 

Feature AEL FWL McREL NCREL NE/I NWREL RBS SEDL SEIL 



I. EVALUATION QUESTIONS 
Criterion questions 
A. NEED STRATEGY 
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Purpose Questions 
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Improve program 
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P 


P 


p 
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OM 


OM 


13. 


Improve SA SA 


SA 


SA 
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SA 




SA 


SA 


14. 
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SD 
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15. 
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C 


C 


16. 


Knowledge 


K 
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II . 


MODEL/IMAGE 


















1 . 


Systems/C I PP* SY 






CIPP* 












2. 


Quality contro QC 


QC 










QC 


oc 


QC 


3. 


Strategic plan 


SP 


SP 














4 . 


Goal -free eval 


GF 




GF 












5. 


Instlt renewal 


IR 






IR 


IR 




IR 





♦Context, Input , Process , and Product mode 

**NWREL addresses several org/mgmt issues dealing with its full range of 
cont racts , but this is beyond the range of this OERI self assessment . 

tt MrREL data from 1986 SA report. Final 1987 SA report not available. 

ttttThe NWREL SA report deals with their 193 contracts of which the OERI 
contract provides 49% 
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Those in the first group focus on a criterion of success, such as 
relevance or efficiency, 10 of which have been identified. Those 
in the second group of six deal with the purpose or use to be made 
of the SA and are concerned mostly with various kinds of desired 
improvements: improvements :o the lab program, to lab organization 
and management, to SA, etc. Within limits, given criterion 
measures might be used for different purposes. 

Further examination of the criterion questions reveals that 
most of them can be related to the three dimensions of our strategy 
cube (see Chapter I), plus two dealing with organization and 
management at a more general level. In Table 4 the evaluation 
questions have been classified as criterion or purpose questions, 
with the criterion questions further subdivided into the strategy 
or management dimensions to which they are related. 

Evaluation Questions for Educational Need. The first strategic 
dimension deals with educational need or policy interest. 
Relevance 1s a criterion that addresses this strategic dimension 
and ties the SA back to needs assessment. It asks in the SA 
context whether the lab 1s continuing to support programs that are 
directed toward the needs of the region. Although not all labs are 
shown as highlighting this criterion 1n >:heir 1987 SA reports, 
probably all do so either in their needs assessment reports or in 
their three to Mve-year plans. Those shewn in the table have a 
formal method of classifying areas of program focus and matching 
these against needs profiles. 

Evaluat 1 on Quest 1 ons for t he I mprovemen t St r a t egy « A number of 
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the other criterion evaluation questions, such as quality and 
efficiency, might be applied to either the improvement strategy or 
the interorganizat ional strategy. They are classified the way they 
are in Table 4 because this is the way they seem to have been 
Interpreted by most of the labs. (NWREL addresses both the focus 
and quality of each of its five institutional functions, which are 
at the level of our strategies). It might be useful for the labs 
to go back and ask if some evaluation questions might be applied to 
both strategies (e.g., the quality, effectiveness, and impact of 
both Improvement: and interorgan izat ional activities^. 

The improvement strategy is represented by several evaluation 
questions. Seven of the labs have instituted qua 1 i ty assurance 
systems (internal, external, or both). (No doubt the other two 
have at least an internal editing review, but the SA reports do not 
mention a quality control function.) They range from routine 
editorial review to internal and/or external technical peer review 
of papers and products. Some labs engage their boards and/or iheir 
advisory committees in the process. Data are collected in a wide 
variety of ways , including surveys of users , feedback forms 
collected after forums and workshops, and evaluation research 
conducted either by lab staff or subcontractors. Quality is 
assessed through judgments based on impl icit or expl icit technical 
or professional standards and is conceptually distinct from impact. 

Utility is a more complex concept than *It may appear at first. 
It is most commonly measured by asking clients whether they find a 
product or service to be useful . But is this an opinion or o 
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judgment based on use? Examination of SA reports reveals that the 
data c i ted often represent judgments of potential usefulness , but 
as an opinion it overlaps with c 1 i en t satisfaction , which we have 
shown as a separate criterion. No labs seem to have identified 
utilization as a criterion in their design, although at least two- 
actually study patterns of utilization by primary and secondary 
clients through tracer studies (AEL and SEIL). Being useful also 
implies something about impact or ef feet i ven ess , another criterion. 
It seems to me that the most analytically distinct concepts are 
use , cl lent sa t isf ac t ion , and impact , and t hat utility is some 
murky combination of the three. 

Impact , sometimes termed "effectiveness," would seem to be the 
ultimate criterion for lab assessment, but this is one of the most 
controversial issues in the laboratory program, with respect to 
both self-assessment and OERI evaluation. One problem is the 
difficulty in distinguishing lab impact from other influences. 
This proolem is compounded by the fact that OERI, in attempting to 
leverage its scarce resources, has mandated the "with and through" 
or indirect strategy, yet seems to demand evidence of impact at the 
classroom level. At the same time, education is acknowledged to be 
a loosely coupl ed system or configuration (see Chapter I above) in 
which ideas and products do not flow smoothly from one organization 
or position to another. 

It may be hel pful to give some ex amp 1 es of impact data 
collected by labs in FY 1987. As mentioned earlier, there is a 
general feeling that it will be more appropriate to measure impact 
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toward the end of the five-year contract period. Nevert he! ess , 
some impact data relating to the improvement strategy are 
presented, which in many cases tend to be simple counts coming out 
of document at i on systems , such as the number of documents 
distributed, the number of conference participants, etc. Such 
indicators tend to fa!"' considerably short of measuring educational 
improvement ; t hey are c 1 oser to being a measure of effort or 
act iv ■ ty . 

Other assessments come closer to the mark. RBS , for example, 

has staff identify the type of impact that can be expected from 

each project or activity and then asks clients tq rate the product 

or activity on those impact criteria. The potential impact areas 

are (RBS Sel f -Assessment Report, FY 87, p. 81): 

o Increase awareness in particular area 

o Provide input for decisionmaking 

o Increase networking or collaboration 

o Provide products and materials 

o Improve organizational climate 

o Improve planning, administration and management 

o Improve classroom teaching 

o Increase staff capability in other areas 

o Improve instructional resources 

o Increase student learning 

o Increase parental involvement in schools 

It should be noted that these are client ratings and not direct 
measures of impact. Note also that all of these except one 
("increase networking or coll aborat i on M ) are indicators of the 
impact of the improvement strategy, not the linkage strategy. They 
seem to be a mixture of proximate and more ultimate impacc 
indicators. (The "provide" items may not be measures of impact at 
all . ) 
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More direct measures can be found in tracer studies and special 
studies , which general ly means an evaluation research project 
directed at a specific project or product. Both AEL and SEIL had 
their sel f -assessment subcontractor use a tracer methodology to 
study the spread and use of lab products. 

NWRE 1 is testing a conceptual framework that posits a six- 
stage continuum of Impact as follows: 
o User sat i sf act i on 

o Acquisition of knowledge and ski 1 Is 

o Use 1n planning 

o Use in implementation 

o Indirect outcomes 

o Direct outcomes 

Note that this framework defines user satisfaction, which we 
had listed as a separate criterion , as stage one of impact. This 
schema was tested in .a case study of evaluation by NWREl'S Center 
for Performance Assessment and appears to hold promise. 

While all labs express their desire to measure impact and 
provide some data that are at least proxies, they approach the 
Issue with understandable caution. Given the problems of multiple 
causation in social phenomena, the methodological and theoretical 
tasks of screening out other influences to measure lab impact, 
especially more distal impacts, are considerable if not 
1 nsurmount abl e . 

We have listed cl lent satisfaction as another evaluation 
question because it has been identified separately in some of the 
suggested guidelines, but, as noted in the NWREL scale, it can also 
be considered as one of the types of impact. Questionnaire surveys 
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and telephone interviews are used by a number of labs to el icit 

expressions of satisfaction. 

Evaluation Questions for the > Linkage Strategy . Although the 

development of appropriate linkage strategies, particularly for 

working "with and through" other ragional organizations, has been 

identified as one of the cornerstones of "Jab operation, the 

specif ication of related evaluation questions has been relatively 

weak. The one most on target appears not in the RFP discussion of 

SA but as one of six questions posed by the OERI evaluation: 

Does the laboratory have productive working partnerships 
with organizations in its region whose cooperation 
1s required to accomplish mutual objectives? 

Even this question is somewhat limited in that partnerships is 
only one mode of relationship, although 1t might be interpreted in 
a more generic sense. Several labs do list the organizations they 
work with and the networks thsy have helped to form, but the 
* quality of those relationships is often difficult to judge. In 

Table 4 we have used the more generic term " interorqanizational 
arrangements " to include all evaluation questions that focus on the 
establishment of effective partnerships, collaborations, networks 
or other working relationships. Only four labs seem to have such a 
focus (FWL, NE/I, NWREL, and RBS). 

NE/I shows an explicit concern with assessment of the linkage 
strategy. This emerges as a second tier of evaluation questions 
appearing under the heading of laboratory effectiveness and impact : 

within the framework of [the five overarchirg goals chat 
frame and structure the work of the lab], Impact within 
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the region can be thought of as successfully developing 

Hnkao »s; serving as a clearinghouse of improvement 

resources and knowledge; Infusing knowledge from the 

research and development community Into reg ional 

activity ; impl ement 1ng dissemination and assistance 
activities; and supporting school improvement efforts. 

(NE/I Year 2 Sel f -Assessment Report, p. 16). 

These questions are then addressed under the following headings of 

the report (ibid,, pp . 16-20): 

o Conven 1ng , f acil 1 tat 1ng , and empowering 

o Increasing ca pa city 

o Broadening perspectives 

o Disseminating and promoting research 

NWREL identifies an "institutional function" that corresponds 

to the linkage strategy: "To what extent does NWREL promote 

communications and linkages among constituents and policy makers?" 

The focus question 1s f "Does NWREL encourage an exchange among 

regional networks of ..constituents?" This question is addressed in 

relation to five "categories of networks": 

o NWREL's Board of Directors Program Committee 
o Program pol icy boards 

o Advisory committees for NWREL's areas of emphasis 
o Spec ia 1 target popu 1 at i on 3 

o School improvement organizations throughout the region 

The quality question 1s, "Has NWREL enhanced the leadership capa- 
bilities and linkages of Its constituents?" The lab suggests the 
possibility of applying the Impact continuum mentioned above to 
this question. 

Two other evaluation questions relating to the linkage strategy 

appear 1n several forms and deal with client contact and awareness: 

o Are lab products and services reaching intended 
audiences? 
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o Are constituent individuals and organizations in the 
region aware of the laboratory, ^ts program, and its 
products? 

The second of these is a bit of a sore point because of the 
"with and through" strategy. It is difficult to maintain lab 
visibility for products and services that are usually provided by 
an Intermediary. This may be one reason that most labs try to 
retain some element of direct service. Only a few labs address 
these questions. 

Evaluation Questions fop Organization/Management . The fourth 
type of criterion question concerns measures of management 
performance that are more general in nature and not tied to lab 
mission. Two have been identified. 

Efficiency . a common managerial standard, is difficult to 
measure 1n the lab context. Even though it is identified as a 
criterion in the RFP, no lab has actually attempted to measure it. 
RBS includes it in its design but has postponed measuring it until 
1988. They define efficiency in ^^rms of "whether [the] 
laboratory's programs, services, and products are performed in [a] 
cost-conscious manner with [a] minimum of waste and duplication of 
effort" (FY 19 8 7 Self Assessment Report , p. A~2 ) . This 
determination is made by the Executive Director and the 
Institutional Review Team by reviewing the laboratory's management 
systems, project plans, and project data base. To develop some 
measures of efficiency that could be used for cross-lab comparisons 
might require that OERI sponsor a special R&D project. 

The second managerial criterion concerns the lab's capacity for 
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conducting self-assessment . 



This criterion is mentioned only in 



the OERI evaluation plan as a possible optional SA issue. Four 
labs provide a discussion of their SA capacity, and, as we shall 
see below, a number have a goal of using the SA process to improve 
their SA system. To obtain a third-party perspective on their SA 
efforts AEL and SEIL employ external evaluation contractors and FWL 
and NWREL engage external advisory panels. Both McREL and SEDL 
used reports of OERI site review teams to modify their SA systems. 
Perhaps some kind of measure of capacity could be derived from a 
consideration of the mechan isms empl oyed . 
Purpose Questions 

The other major type of evaluation question we have termed 
"purpose questions." Each identifies a purpose to be served without 
specifying the criteria to employed. The other side of the 

purpose "coin" is the us to be made of the SA data. In some cases 
the Hdta from criterion questions can be used, but in others 
additional measures would need to be identified. The data in Table 
5 represent a mixture of stating the purpose in the evaluation 
design and citing examples of the use made of SA data. 

Improve Lab Programs . The improvement of lab f unct i on ing is 
probably the central purpose of SA . This purpose is served at 
project, program, and institutional levels and includes both the 
fine-tuning of current work und the planning of future work. For 
the most part the use of documentation and evaluation data 
concerning specific projects and activities for the formative 
evaluation/improvement of products and services is too detailed to 
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be ^rorted. Most of this occurs at and is the responsibility of 
1 ower echel ons and is part of day- to-day opera t ions . Some 1 abs 
report this kind of detai 1 , whi le others prefer summaries and 
interpretations based on it. At the program and institutional 
levels labs use SA findings to continue, discontinue, restructure, 
and plan programs, reorganize the structure, and change strategies. 
Some examples: 

SEDL found that its Task 3 activities, which were 
essentially the dissemination of its programmatic theme 
outcomes to state dec^sionnakers, were too proactive and 
not in a position to respond to the emerging needs 
expressed by clients. As a result they decided to begin 
a wholly new program (ED-AIDE) 1n 1988-90 thai: 
represented a 180 degree change, I.e. to a program 
featuring 1 imlted short -term/rapid-response act i vit 1es 
for regional partners and appropriate resource 
development activities. 

RBS had operated Its Task 3 program on a state-by-state 
basis. However, feedback from state liaisons Indicated 
Increasing awareness of common or overlapping concerns. 
Recommendations have been made to disseminate 
Information on project acti.'^ties 1n one jurisdiction to 
the other four, and to promote project activities that 
encourage and Involve multi-state participation. 



Improve Lab Organization/Management . Several labs had external 

peer review teams or contractors that examined Issues of lab 

governance ^nd administration. Including staff development and the 

'mprovoment of sel f- assessment itself. Examples of recorrmendat i ons 

and changes emerging from these processes are: 

NE/I expanded the Executive Committee of !ts board 
from 1 f , to 12 £nd from two chiefs to three in order 
tc address concerns about the level of board and 
especially chief Involvement. 

The FWL Evaluation Advisory Panel recommended that 
''You should stop trylr.g to produce good evaluation 
data , beyond the requi red mini mums , across all 
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projects. Rather, you should worK intensely with two 
or three projects that seem to have a high level of 
interest or capability to work on formative 
evaluation or documentation case studies. Second , 
you should develop a minimal set of institutional- 
level accountabll 1ty Indicators . " (FWL 1987 Self 
Assessment Report , p . 4 8 . ) 

Staff development is listed as a purpose of SA only 
by NCREL, but no examples are provided. Some 
recommenda t i ons f st rengt heni ng staff devel opment 
emerge from other activities in other laboratories 
(FWL and NE/I) . 

Comply with Contract Requirements . Contract compliance is a 
purpose more appropriately served by the annual reports, but it 
appears to be a central feature of the SA reports from AEL, McREL , 
SEDL, and SLIL . Some minimal pragmatic learnings can be gleaned 
from examples of "we were unable to do X because of barrier Y. M But 
endless spelling out of program, project, and activity workscopes 
and outcomes can be deadly. OERI might consider shifting to a 
management-by-except ion mode to avoid this difficulty (as well as 
des 1gnat1ng the annual report as the place to report com pi 1 a nee 
Information) . 

Increase General Knowledge, The RFP gives some salience to the 
goal of building the knowledge base concerning "effective 
strategies for 1mpi . ving education through carefully designed 
studies of how Its own dissemination and improvement efforts are 
working" (p. 21). Only four labs state this as a purpose of their 
SA efforts, although more than four actually report on "learnings" 
from SA activities. Some even commissioned special evaluation 
studies with external contractors. 

An Interesting gap exists between the empirical findings of SA 
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activities on the one hand and statements concerning the knowledge 
gained on the other. Most of the reported data-based learnings are 



major effort to sift through them to find patterns and 
consistencies . However , sever a 1 1 abs , notabl y FWL and NE/ I , have 
provided thoughtful discussions of major strategic issues that are 
Informed by experierce but not clearly derived from SA studies. 
Such discussions are valuable and should rot be discouraged. But 
there 1s a desperate need for some theoretical and methodological 
frameworks that would enable the design of SA studies that might 
lead to more general principles. Dunn's work on the ADD modfe 1 and 
a theory of applications is an example of what is needed (Dunn 1986 
and 1987). 
Model /Image, 

While some labs approach SA and its related evaluation 
questions 1n a somewhat ad hoc manner, others refer to a more 
unifying nodel or image. Some of these appear more in the design 
than in the execution, but they give a further clue as to SA 
aspirations. Although we have Indicated at least one model or 
image for eve^y lab, some had to be inferred. In some cases an 
explicit frcnework is laid out and followed, while 1n others the 
Image may be part of a background discussion. Although in most 
cases there is only one such model or image, FWL has referred to 
five. 

Systems /CI PP. It is fair to say 'that all labs use systems 
concepts at least implicitly, ar.d that systems concepts are 



pragmatic and ad hoc bits of craft knowl edge . 



It would 



take a 
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implicit 1n the RFP . T ley all employ systems diagrams that show the 
flow of information between units, with feedback loops, etc. AEL 
uses an explicit input/output model that shews lab output as an 
input to service del i very organ izati ons . NCR EL incorporates the 
CIPP model (standing for context, input, process, and product 
evaluation) in its design. (Note that context evaluation 
corresponds to needs assessment . ) 

Management Qua! 1ty Control . Virtually all labs view SA as a 
tool of management for maintaining control of operations and 
assuring accountability. In some labs this perspective is the 
figure and in others the ground. Whether there is an explicit 
three-level organizational design, project Information is reviewed 
by program heads, and program information 1s reviewed by top 
management and the board. 

Strategic Planning. We have said that the overall planning 
model under which the 1 abs operate and of which SA is a component 
is st rat eg 1c pi anning . In the SA context it comes into play 
primarily in addressing the relevance evaluation question. 

Goal-Free Evaluation . The models or Images discussed so far 
all represent forms of goal-based evaluation. The need for 
accountability under government contracts dictates at lease some 
use of such models. Some labs (FWL and NCREL) also aspire to use 
goal-free evaluation or Its variants, but how this view is 
Implemented 1s unclear . 

Institutional Renewal . Finally, "institutional renewal" is the 
term we have given to an image in four labs. Tnis image seems to 
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be related to the fact that SA is part of the planning cycle and 
through successive iterations of that cycle the institution is 
continuously renewing itsel f . 

Mechanisms for Implementation of Sel f -Assessment 

The mechanisms for implementation of SA are the ways of 
collecting SA data and the organizational arrangements for review 
and decisionmaking. It is helpful to examine these mechanisms at 
three different organizational levels: the institutional, program, 
and project . (Some labs incorporate this three-level schema in 
their SA designs; for others we have Inferred 1t.) Lab profiles are 
shown 1n Table 5. Some labs say that they have three-level SA 
systems but have limited the SA report to institutional level 
assessment. However, some program and project level information is 
contained in these reports. 

Institutiona l Level . SA is employed by top management and the 
lab board to review ** u e overall program in relation to the lab 
mission. Both exterr,! and Internal mechanisms are employed. 

Since 1980, RBS has used a three-person external peer r evi ew 
team , the Institutional Review Team, to provide an outside 
perspective on lab functioning. The team meets twice a year and 
focuses on different aspects of the lab as the need arises. It had 
a constant membership for several years, but will begin rotating 
the membership 1n 1988. One of the outcomes of the SEDL SA process 
was the decision to add an external peer review team to its SA 
system 1n 1988-90. To emphasize a more reflective look at 
1nst1tuti ona 1 improvement , the team was given t he interest ing name 
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Table 5. Mechanisms for Implementation of FY 1987-88 Sel f -Assessment Designs 
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IR* Team, standing for institutional review, reflection, and 
renewal . A different kind of external peer review is provided by 
the OERI site review teams. A number of labs report the comments 
of these teams and t hei r responses to them . A prel iminary draft of 
the McREL 1987 SA report took the form of responses to the comments 
of its OERI site review team, but this report is not the one cited 
in our tabulations. 

In other castas , subcontracts are let to external organizations 
to perform specific evaluations. Both AEL and SEIL use this device 
and employ this method for obtaining a meta evaluation of the lab's 
own SA efforts as well as evaluations of lab management and 
programs. The SEIL subcontract also examines lab governance. In 
both cases, subcontractors conducted tracer studies of specific lab 
programs. 

Another external device is use of an externa 1 advisory 
committee specifically to advise the lab on Its self -assessment 
design and operation. FWL employe such a group. In operation this 
group seems to go beyond this charter at times to function more 
like a peer review team for the total lab operation. NWREL employ* 
a "consultant review panel M but provides little Information on its 
operation . 

S tate advi sory commi ttees , which provide needs assessment 
input, often fLnction as an SA mechanism as well. Some labs use 
thes committees to review lab documents and products as part of the 
quality assurance system. 

Other mechanisms are Internal to the lab. Four labs mention 
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some form of top management counc i 1 consisting of the executive 

director and program heads that advised the executive director. 

The mechan isms used by boards vary and are not always easy to 

identify. Where the lab is part of a much larger organization 

there is usually a subcommittee with special lab oversight 

responsibilities. FWL has devised a specific technique modeled 

after financial investment models called portf ol io review : 

To accompl ish the review of the entire portf ol io , a 
four-part form is used. In part one, the portf ol io is 
scanned and. . .seven questions are answered pertaining 
to: risks; probability of payoff in terms of affecting 
school or classroom Improvement ; benefit -to-cost ratios ; 
helping the Laboratory to develop or improve its 
capabilities to work with and through other agencies; 
Improvements in the capacity to serve as a regional 
laboratory; successful cost-sharing; and maintaining a 
good reputat ion with key constituencies . 

...the primary purpose of these questions is to promote 
discussion ainong staff that engender better common 
understanding of the strengths and weaknesses of the 
total program . . . 

Part two of the review 1s a scan of developmental areas . 
This ccan responds to six questions: identify potential 
act 1vit 1es that wou Id help FWL res pond mot'e effectively 
to emerging needs; which client groups are served least 
wel 1 ; assistance strategies that are most strongl y 
represented in the current program; promising technical 
assistance strategies that are most lacking; the most 
product 1ve or promising things the Laboratory is Hoing 
in developing an effective and efficient R&D support 
capability; and the most cr itical or important 
capabi 1 it ies that are needed to be more effective in 
meet 1ng reg iona 1 and nat 1onal ex pec tat ions for a 
regional laboratory. 

Part three 1s an evaluation of the total portf ol io . 
Again, a seven-point agreement rating scale is 
employed. . .ranging from low tc high agreement. The 
Instructions are, "Considering the present set of 
activities contained 1n the Regional Programs portfolio, 
how would you rate the portfolio on the f ol lowing 
character 1st ics? M [fifteen characteristics] 
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Pa-t f our of the portfolio review is a contingency 
planning exercise , which calls for responses to... three 
questions [hypothetical ten percent increase in funding , 
ten percent cut, and] what do you see as being the most 
promising things we could do to increase the return on 
investment of our present set of resources? 
(FWL Second Annual Self Evaluation Report, pp. 11-13, 
Italics in original). 



This illustrates how a technology can be developed to implement 
aspects of SA . 

Seven labs have designated a specific internal eval uator 
posi t ion in the office of the executive director as the locus of 
responsibility for lab SA. In others, this is a responsibility of 
some other official. Specific sub-tasks may be delegated to line 
units . 

There is a Task 5 theme for SA , for which SEDL is the lead lab 

jnd 1n wnich all labs participate. Meetings of staff responsible 

for SA have been held and labs have shared their SA reports with 

each other. A report on FY 1987 activities lists four objectives 

of this of the collaboration (Hoover 1988): 

o To strengthen each individual institution's 
evaluati on efforts by sharing expertise and ideas 
concerning laboratory evaluation purposes, designs, 
Implementations , and results 

o To assist the planning and implementation efforts of 
OERI as it carries out its responsibilities in 
Laboratory evaluation by providing Input on effective 
and efficient ways to collect and use laboratory 
eva luati on i nf or mat ion 

o To provide a broadened perspe. cive on laboratory 
operations by synthesizing evaluation information on 
the effectiveness of selected major laboratory 
st rategi es 

o To enhance planning for subsequent laboratory efforts 
by monitoring current trends 1n laboratory evaluation 
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work and reporting on accomplishments of the 
col laborat i ve 

Program Level . For the most part the designs say rel at i vely 
little about special mechanisms at the program level. Basically SA 
at that 1 evel appears to consist of review by the program head and 
his or her project directors of project level data. A number of 
the SA reports are organized by program . This is probably the 
primary level for .ie conduct of compl i ance reviews . Program 
advisory committees are a specific mechanism used by three labs. 
But it 1s hard to find mechanisms for determining whether the 
programs are mo^e than the sum of their project parts. 

Project/Activity Level . Mcst SA data are collected at the 
project/activity level. Labs attempt documentation of many aspects 
of their operation. _ For example, they maintain logs of phone 
calls, document requests, field contacts, etc. As mentioned 
earlier, AEL listed 28 forms of documentation; but not all labs 
provide such lists, and no attempt will be made here to profile all 
documentation activit ies . 

We wi 1 1 attempt a more specific prof il ing of evaluat ion 
mechanisms. Most labs have some form of internal qua! i ty assurance 
that reviews reports and products for editorial and technical 
quality. Five have external qua! i ty assurance systems is well. 
For the most part this consists of using rating forms, which are 
also used as feedback mechan 1 sms to determine the reactions of 
participants in workshops and in other meetings. Such mechanisms 
may provide important formative data to lab writers and developers 
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but are rarely of general interest. 

Special mention needs to be made of tracer studies and other 
special studies that use more sophisticated designs to evaluate 
specific projects. Both AEL and SEIL employed outside subcon- 
tractors to conduct tracer studies to determine whether impacts 
could be traced from primary clients to secondary cl ients (both 
with disappointing results). SEDL attempted to build tracer data 
into its documentation system, but had little success. 



Conceptual Issues 

This chapter has covered a lot of territory. The implicit 
strategic planning model has included needs assessment, regional 
capacity, and self-assessment . The position taken here is that 
planning is the best term for this complex set of operations 
focused on the use of Information for decisionmaking. One problem 
encountered is that each of the components of planning has tenaed 
to be " Imperial istic , " i.e., each component concept tends to expand 
to include territory claimed by the others. For example, needs 
assessment, as conceptualized by Kaufman, has been expanded to 
include the entire territory we have termed "planning." Similarly, 
evaluation has been used for the same purpose in the CIPP model . 
In the present analysis sel f -assessment starts as a discrete 
component of the strategic planning process, but whun one examines 
the sel f -assessment reports they seem to include far more than 
sel f -assessment "properly so-ca 1 1 ed . " 



Comments on Planning 
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Under these circumstances, it is not surprising to see certain 
kinds of information appearing in multiple reports (i.e-, plans, 
needs assessments, sel f -assessments , and annual reports). The labs 
and OERI need to work together to sort out the several pur poses and 
reports so that each kind of information has its proper place Th« 
need is to narrow the scope of each component concept to avoid 
overlap and exclude extraneous matters. In the sections below I 
will comment on eac K of the concepts and suggest some steps that 

might be taker to achieve this. 

r 

Needs Assessment 

On the whole, labs seem to do a reasonably good job of 
assessing the needs of their regions. A variety of methods have 
been tried and many produce useful Information. No doubt lab staff 
members have an even -better "feel" for their regions than emerges 
from their formal methods. 

Nevert hel ess, there are improvements that might be made . One 

need is some conceptual c , ar i f 1 cat ion as to the various possible 

types of need (see above). Another is to recognize that needs 

exist at different levels of generality. It is useful to 

distinguish three levels of generality, i.e.: 

o Level of educational functions: e.g. Improve teaching 

o Level of choice : e.g . , improve beginning teacher 
1 nduct 1 on 

o Level of design decisions; e.g., best ways to des ign a 
mentor teacher program. 

The 1 eve 1 of edu ca 1 1onal functions 1s largely inappropriate for 
needs assessment. Every lab should have the capacity to provide 
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services for each of the major functional areas of education. (I 
say "largely" because there is the secondary consideration of the 
balance of effort among these areas.) At the other extreme, the 
level of design decisions, choices should be based on the best 
ava 1 1 abl e research and craft know 1 edge ; i.e., the dec i s i ons are 
technical decisions. Needs assessment should be focused on the 
middle level of generality, i.e. the level at which values and 
Interpretations of the regional context operate . 
Regional Capabl 1 1tv Assessment 

As indicated above, this was the most poorly implemented 
component of the planning model. No doubt the labs have better 
Information on this topic than appears in their re orts, if only 
the personal knowledge of the staff. However, we cannot understand 
whether this component is contributing to the planning function 
unless this knowledge and its application are made explicit. While 
an important start on the met ho do 1 ogy of this assessment was made 
by the ARROE Study (Frankel et al 1979), elaborations are needed to 
make the measu rement of organizational ca pa city commensu rat e with 
other elements of the planning model, i.e., needs or policy 
domains , improvement capacities , and 1 inkage capacities. 
Sel f -Assessment 

The sorting out of mul tiple purposes needs to take pi ace 
primarily 1n the sel f -assessment component . A number of the 1 abs 
used a three-1 evel model that ident if ied SA quest ions and 
mechanisms at Institutional , program, and project levels . 

Not all these levels are appropriate. The 1986 SA reports 

ERIC i*o 



CHAPT III PLANNING Page II 1-47 

Included a much formative evaluation data from the project level . 
The older labs hove a strong background in development, and they 
collect a significant amount of data aimed at making design 
decisions for fine-tuning product development . However , even here 
there are problems. For the most part these studies have yielded a 
large amount of craft knowledge that has yet to be codified and has 
only limited potential for bui lding a knowledge base of more 
general application. Most evaluation is "black box" evaluation; 
e.g. clients are asked to evaluate some product whose 
characteristics have not be specified and for which the strategy 
for bridging organizational space has not been described. Where 
are the studies that compere the effectiveness of two workshop 
formats for the same type of audience or of one format with two 
types of clients? Where are the studies that compare two ways of 
organl z ing networks? 

It was recognized that most of the formative evaluation data 
sup pi ied in the 1986 SA reports was not very useful for SA 
pur poses , and most 1 abs compl ied with a request to reduce the 
amount of this detail 1n their 1987 reports. The next step would 
be to recoc ize that formative evaluat ion data are part of the 
development process and should be excluded from sel f -assessment . 
The relevant SA question would be: "Do we have adequate methods of 
formative evaluation 1n place in each program?" 

Similarly, a great deal of the documentation data (phone logs 
etc.) may provide a useful descriptive context, but they are either 
grist for the formative evaluation mill or mere counts that require 
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interpretation if they are to be understood in ^ sel f -assessment 
context . 

Also, detailed contract compliance data are not the stuff of 
thoughtful consideration of lab mission and strategy. They bel ong 
in quarterly and annual reports, not sel f -assessment reports , and 
there only on a management-by-exception basis. 

It 1s recommended that sel f -assessment (reportable to OERI) be 
limited to the Institutional and program levels and focused more 
sharply on two aspects of lab operation: issues of organization and 
governance (e.g., is the lab organized in accordance with and 
operating within its mission?), and Issues of strategy, (i.e., 1s 
the lab responding to the needs of the region, is it using the best 
improvement and linkage strategies, and is 1t contributing to the 



with few exceptions, lab SA is not focused on the strategic 
questions of lab operation. (The notable exceptions are the AEL 
study of the study group methodology and the AEL and SEIL tracer 
stud ies . ) 

Another problem is the unavoidable overlap between SA and 
OERI's desire to evaluate the laboratory program as a whol e . Given 
the history of prickly relationships between the labs and the 
government and the labs' strong sense of vulnerability, is a 
surprising amount of candor appears in their reports. While it is 
jnderstandabl e to want to put a good face on everyt hing , t he 1 abs 
acknowledge numerous problems and weaknesses. 

More candor might be forthcoming if more of a separation 



knowledge base concerning these strategies?). 



It appears that, 
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between sf, I f -assessment and OERI evaluation could be achieved. One 
suggestion would be to try a different set of labels. "Evaluation" 
and "assessment" are terms that automatically trigger defensive 
mechanisms and stifle the intended learning and renewal purpores. 
Problems might be avoided by legit imi zing a laboratory research 
program to study laboratory operations and functions, thus helping 
to move the labs away from their excessively empirical approach 
toward a more theoret 1c a 1 1 y rel evant agenda - The research fields 
•involved include 1 nterorgani zat ional theory, knowledge utilization, 
dissemination, and social change. Labs should be contributors as 
well as consumers of research 1n these fields. 

A limited amount of work of this kind has been done, e.g., 
AEL's examination of the effectiveness of study groups as an R&D 
methodology and the SEIL tracer studies. 
Reprise on Rational Planning 

Whi 1 e pi an rn'ng 1s inevl tabl y a proactive approach to probl ems , 
labs are also expected to be reactive. More than that, as 
sociologically marginal institutions they are constantly in the 
position of having to renegotiate their role in the regional 
Infrastructure. The reports contain many 'examples of how, for 
example, a whole new relationship to an SEA had to be worked out 
when there was a change in chiefs. Such events create havoc with 
carefuly laid plans. The entire planning process needs to be 
examined to determine which parts nave heuristic value and which 
ones play only ritualistic roles. 
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CHAPTER IV 

WORK WITH STATE LEVEL DECISIONMAKERS 
ON SCHOOL IMPROVEMENT ISSUES 

Introduction 

Ever since the beginning of the educat ional reform movement 
triggered by the report of the National Commission cn Excellence, A 
Nat ion At Risk , education policy issues have been much more salient 
at the state level. Many governors and state legislatures have 
placed educational improvement high on their agendas, and the 
National Governors* Association has Issued its own report (National 
Governors* Association, 1986). Probably the key to the new focus 
on educational issues has been the realization of the Importance of 
a strong educational system 1n the inter-state competition for 
economic development- A number of states have adopted 
comprehensive school improvement programs and virtually all states 
have undertaken specific reform initiatives that go beyond the 
usual financing issues. State level professional associations, and 
groups representing business, labor, and parents have been active 
in the accompanying policy debates. 

However, the states vary greatly 1n their capacity to deal with 
these Issues . Larger states such as New York and Cal if ornia have 
sophisticated staff in thei r SEAs and pol 1cy aides 1n the 
governor's office and legislative committees, but many others are 
ill-equipped to deal with the problems of policy formulation, 
implementation, and assessment. Further, states also vary greatly 
1n the availability of non-governmental resources from such 
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entities as universities and research firms. 

It is in this context that we examine Task 3, which requires 
that labs "work with state-level decisionmakers on school 
Improvement Issues." Three Illustrative tactics are listed in the 
RFP: technical assistance. providing issue-specific analyses and 
syntheses, and facilitating communication amorg state-level 
dec is ion makers . 
Federal Specifications 

Unlike Tasks 2 and 4, which Involve the "with and through" 
strategy, Task 3 states that the lab will work with state-level 
decisionmakers. This may or may not be a distinction without a 
difference. State-level groups are Intermediaries with regard to 
school improvement at the local level. However, 1f the focus 1s on 
reform at the state policy level, state-level decisionmakers can be 
considered direct clients and the labs will be judged on the basis 
of their delivery of direct services for Task 3. 

But what does 1t mean to "work with" state-level decisionmakers 
on school Improvement issues? It 1s important to note that policy 
making 1s a social process while policy analysis is an Intellectual 
activity (Dunn, 1986). Some observers have questioned the value of 
professional social Inquiry in affecting the policy process 
(Lindblom and Cohen, 1979). Perhaps this is due to the primitive 
state of policy analysis and the lack of a sound theory of 
applications (Dunn, 1986). In any case, the mission of the labs 
under Task 3 1s to assist state-level policy-makers to make better 
use of knowledge at various stages of the policymaking process. 
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State Education Pol Icy Consortiu m 

In 1983 NIE funded a two-yean project with the State Policy 
Consortium, a group made up of the Council of Chief State School 
Officers, the Education Commission of the States, the National 
Association of State Boards of Education, the National Conference 
of State Leg 1s 1 atures , and the National Governors f Assoc 1 at ion . 
These national organizations count as th*ir members many of the 
principal state-level decisionmakers who are the targets of the lab 
Task 3 efforts. Among other things, the project was designed to 
"...[Improve] our understanding of the sources of Information 
policymakers rely on, and of effective strategies for disseminating 
research to policymakers" (Cohen, 1985a, p. 1i. See also Cohen, 
1985b; Fuhrman and McDonnell, 1985c*; & 1985b). In addition the 
study made recommendations on the roles of the Consortium 
associations, the regional labs, the Center for Policy Researc. in 
Education, and OERI . 

The project was completed in 1985 at the time that the new labs 
were being organized, and a meeting was held involving Consortium, 
lab, and OERI staff. As a result, the labs had the results of this 
project available at the time their new contracts began (although 
not necessarily at the time the plans were written). 

A major finding of the research was the strong interest 
expressed among all states and stakeholders in having aval 1 able 
Information on the specific policy Issues being addressed by other 
states. The project had collected a 11st of state policy "data 
bases , M many of which were coll ec ted by the Consort ium 
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organizations, but many of these consisted of ons-time studies. 
The report made recomnnendat i ons concerning coordination and 
improvement in the collection of useful data by the member 
organizations. Three roles for the labs were suggested (Cohen, 



o Cooperation in devel oping in -depth descr i pt ions of 
state pol 1cy provisions , on a 50-state basis , in 
selected pol 1cy issue areas (Initially rejected by labs 
at meeting with Consortium staff in 1985) 

o Conduct of studies of the implementation and Impact of 
state pol icy Initiatives 

o Creation of an Information base of studies of state 
policy Implementation and impact conducted by other 
agencies and serving policy formulation needs 



Elsewhere labs were urged to adopt dissemination strategies 
that "allow for easy access to Information. low consumption costs 
for the user, a direct application cf Information to specific state 
or policy contexts, and timely receipt of information. Especially 
valuable is the use of structured meetings that encourage an 
Informal interaction between Information users and Information 
providers" (Cohen, 1985b, p. 67). Further, 1n recognition of the 
importance of Instate sources of information, the labs should 
establish or strengthen Instate issue networks. This would involve 
identifying sources of expertise and bringing them together with 
state policymakers. Strengthening the capacity of the State 
Education Policy Seminar- (SEPS), jointly sponsored by ECS and I EL , 
was recommended. Finally, labs can play a strong role 1n 
connecting state policymakers with national and state R&D networks 
and 1n studying local district and school responses to state 
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Improvement Initiatives . 
Methodological Issues 

This is the first place that we encounter a methodological 
issue common to Tasks 2-5, namely the choice of a unit of analysiJ. 
The most detailed analysis could be done if we could identify and 
classify each project or activity conducted oyer the two-year 
period. However, this 1s difficult, if not impossible, for several 
reasons. First, while some activities are discrete units of work 
having finite tinr.e lines and determinable characteristics, others 
are ongoing services of a more open-ended nature (e.g., data base 
maintenance). Second, the amount of information available and the 
conceptual frameworks used vary greatly from project to project and 
from lab to lab. Third, there is no uniform way of structuring 
work into programs, projects, activities, and sub-activities. As a 
result, cruder methods are used here that aoply to the entire task 
area for each lab, Indicating either presence or absence of a 
particular tactic, or sometimes a degree of emphasis (high, medium, 
low). Based on these data, the attempt will be to find patterns 
indicative of broader strategies. 
Organization of Chapter 

The following sections describe lab activities under Task 3 for 
the three strategic dimensions of linkage strategy, improvement 
strategy, and educational need; identify the policy stage most 
frequently addressed; examine the national coordination of state 
pol 1cy activities ; and then present a hoi istic picture of each 
lab's Task 3 program. Comments summarizing the analysis appear at 
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Findings 

This task has the three st rat eg ic dimensions of the cube 
presented 1n Chapter I . 

First, each lab needs to design a linkage strategy by deciding 
who Its primary clients are and how It will relate to thern. (Note 
that such "designs" cJlow for the possibility of opportunistic 
strategies.) A lab 1s expected not only to mediate relationships 
among organizations within the region but to provide access to 
national resources. Lab regional services should complement 
national resources with more context-specific assistance. 

Second, each lab needs to design an Improvement strategy In 
terms of some mix of KPU functions as they apply to selected stages 
of the policy process. 

Third, each lab needs to determine whether It will attempt to 
work across a more or less comprehensive set of policy Issues or 
domains or focus on a limited number of such Issues, and, 1f the 
latter, which ones. 

Cross-cutting these dimensions is a distinction concerning the 
goal of the activity: is the activity intended to bu i 1 d capacity 
for policy analysis (either the lab's or that of some other entity) 
or to do policy analysis? 

Although the focus of this chapter is on TasK 3, one of the 
themes developed under Task 5 (cross lab coordination) deals with 
state policy also and 1s addressed here. 
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State Pol Icy Linkage Strategy 

Profiles of the linkage strategies employed by labs in their 
second contract year are shown 1n Table 6. This table should not 
be taken as definitive because a service not shown in the table 
might be found under another task. In addition, data are derived 
both from plans and from progress reports, and sometimes 
discrepancies appear. Nevertheless. the table is indicative of 
broad patterns. 



Table 6. Profiles of Lab Task 3 Linkage Strategies for Working 
with State-Level Decisionmakers on School Improvement Issues. 

Linkage Laboratory 

Strategy AEL FWL McREL NCREL NE/I NWREL RBS SEDL SEIL 

Component 



1 . 


Col 1 aborators 


















a . 


Regional 


HE 


SPA 


SEPS 


SEAs 


SEA, HE 




ISA 




H 


b. 


National 


CVE 






CPRE 


ECS 


ECS 




CPRE 


CPR 


2. 


CI 1ents 




















a . 


Pr 1ma ry 


CH 


CS 


S 


CS 


CS 


CS 


S 


SG 


CH 


b. 


Range 


N 


B 


M 


B 


M 


B 


N 


M 


M 


c . 


Type 


N 




0 


AH 




0 


0 


0 


N 


3. 


Communication 


















a . 


Approach 


S.R 


R 


S,R 


3 


• 

S 


R 


S 


S 


R 


b. 


Type 


M, R,B 


R 


C,M 


M , R 


M, R 


R.M.P 


P 


B 


R 



1.a. Regional Col laboratorrs : Higher Ed Inst1tu1ons? State 
Prof Assoc; X ntermed ^ at © Service Agency 

1. b. National Collaborators: CPRE , CVE (Center for Voc Ed); ECS 

(Ed. Commission of the States) 

2. a. Primary Clients: Chiefs, SEAs, State Gov't, Cross Section 
2.b. Range of Clients: Narrow, Medium, Broad 

2. c. Type of Clients: Ad Hoc, Network, Organization 

3. a. Communication Approach: State, Region 

3 . b. Communication Type: Meetings , Reports , Briefings , 

Consu 1 ta t 1ons , Products 
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The three major dimensions of the linkage strategy are "three 
C's": collaborators, clients, and communication modes. 

Col laborators . Collaborators are not nearly as significant for 
Task 3 as for Tasks 2 and 4 because the "with and through" strategy 
is not involved. At the national level, the Center for Policy 
Research in Education at Rutgers University cosponsored regional 
policy symposia with NCREL , SEDL , and SEIL, and the Center for 
Vocational Education collaborated with AEL. ECS worked or. projects 
with HE/ 1 and NWREL. Regional col 1 a bora tors include SEAs, 
Intermediate Service Agencies, Institutions of Higher Education, 
and SEPS units (which, of course, are instrumentalities of two 
national organizations, ECS and IEL). 

C 1 ients , The number of poss i ble groups to serve is considerabl 

A list of the major potential clients would include: 

o Chief state school officers 

o Ot her SEA personne 1 

o State board of education 

o State higher education CEO and board 

o Governor 

o Governor f s aides 

o Leg is 1 at ors 

o Leg is 1 at,i ve aides 

o State professional associations 

o State business associations 

o State labor associations 

o Other state advocacy g rou ps 

o Intermediate a gene i es 

o LEAs 

The latter two categories are included because they are sometimes 
involved in lab-sponsored meetings even though Task 3 is directed 
at state-level decisionmakers. 

While there may be a desire to work with all these groups, 
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limited resources, the number of states or other units to be 

served , and other considerations require some choices . T he issues 

that arise regarding client selection are: 

o Which groups are given primacy? 

o Is the range of clients broad or narrow? 

o What form does the cl ient group take? 

Although, in a sense, SEA's are the primary cl ients of all 
labs, the variations on this theme are considerable. For AEL it is 
hard to overestimate the central ity of the chiefs as significant 
others. They all serve on the Board of Directors and collectively 
constitute the Council on Policy and Planning, which serves three 
functions: (1) it has special responsibility for reviewing lab 
projects regularly and recommending to the tfoard any changes in 
overall lab direction; (2) it serves as the Advisory Board for the 
Policy and Planning Center (i.e., the Task 3 program); and (3) 
individually, the Council members are the primary clients of the 
Poliwy and Planning Center. 

For most labs the SEA as a whole is the primary clien'c. RBS 
goes further than any other lab in defining itself as an adjunct to 
SEAs. (For details see the RBS configuration below.) Most labs 
also try to relate to other elements of the state government, i.e., 
the governor's office and legislative committees. SEDL seems to 
have the most balance in its attention to the several components of 
the state government . 

Four labs deal with a cross section of state-level 
decisionmakers that goes beyond state government to include state 
professional associations and various advocacy groups (FWL, NCREL, 
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NE/I, and NWREL ) . (Not that other labs exclude such groups; it is 
a matter of emphasis and balance.) 

This discussion leads into a characterization of the range of 
clients as narrow, medium, or broad. The coding on this factor in 
Table 3 is admittedly impressionistic but takes us a step beyond 
identifying the primary client. AEL and RBS seem to have the 
narrowest focus on state-level decisionmakers, and FWL , NCREL, and 
NWREL the broadest. 

Final ly, the type of cl lent may be a formal organization, 
networks of individuals, or ad hoc collections of individuals. 
Coding of this item is particularly diffi cult becau se t he 
individuals served invariably have roles in client organizations. 
But it makes a big difference whether the individual is involved 
for personal reasons- (e.g, professional development) or 
organizational reasons (e.g., Implementation of a training 
program). As we shall see below, at least one lab made the mistake 
of selecting for a trainer of trainers program individuals who had 
no organizational responsibility for inservice training. 

For Task 3 most labs seem to emphasize the formal organization 
as the client. For example, NWREL has a broad clientele and often 
makes presentations at meetings of professional associations and 
other groups. In a number of cases networks have been established 
among legislative aides in different states or among legislative 
aides, governors aides, and SEA policy representatives. We also 
classified the chiefs involved with the AEL Policy and Planning 
Center as a network, although it might be considered a formal 
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organization. Unfortunately, 1t 1s often difficult to tell from 
the documentation whether those Involved 1n an activity constitute 
a network or an ad hoc group. 

State and regional policy forums and symposia usually have an 
ad hoc partld atlon depending on the topic, but may involve 
continuing networks. 

Communication Mode. The third facet of the inkage strategy is 
the mode of communication. A major difference in communication 
approach occurs between act1 v1t 1es that are organized on a 
sta^e-by-state basis and those that are organized regionally. 
Determining which 1s more appropriate will depend on a number of 
considerations. To the extent that states within a region have 
common problems and Interests, a regional approach 1s feasible; 
where there are considerable differences among the states, dealing 
with them Individually may be required. The^e 1s a continuing 
tension between the need to be responsive to varying needs and 
contexts and the need to hrlng broader perspectives to bear on 
problems that are rare*/ unique. Many labs have opted to work with 
the State Education Policy Seminars (SEPS) jointly sponsored by ECS 
and IEL, and this format requires a state approach. In other cases 
1 abs have cooperated wi t.h the Center for Policy Research in 
Education (CPRE) 1n organizing regional policy symposia. Aside 
froi.i meetings, most direct services such as those provided to 
chiefs and SEAs are organized on a state basis. SEDL has made a 
special effort to provide more or less Identical services to each 
state 1n Its region, but to organize them by state, not by region. 
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(Interview with Institutional liaison). 

Many types of communication are employed. Including meetings, 
reports, briefings, consultations, and products. Mer in ^3 1n their 
various forms (symposia, forums, conferences, etc.) generally 
Involve the widest array of participants. Meetings and reports in 
various "Dunn types" (i.e., policy Issue papers, policy briefs, 
policy news releases) are common to almost all labs. SEDL 1s 
noteworthy for its frequent use of briefings for state personnel 
and Its attemnt tc use an electronic bulletin board (later 
abandoned). Both FWL and NE/I have consulted directly with 
legislative committees on specific pieces of legislation. 
Pol 1cy Improvement Strategy 

The services offered as part of the policy Improvement strategy 
are shown 1n Table 7. ..(The caveats enumerated for Table 3 also 
apply here.) They cover the entire spectrum of KPU activities. (In 
this table an "X" 1s used to Indicate presence of the strategy 
component 1n one or more activities or projects. ) 

Information Services . CI earl y labs see themselves as sources 
of knowledge 1n the broadest sense. In the first instance, this 
takes the form of a number of specific Information services . These 
are separated 1n che analysis from other Improvement activities 
because they are primarily capad ty-bu1 1 ding functions ; they 
provide an Information resource that facilitates the actual 
performance of policy Inquiry. Information services include: 

o Development of several types cr databases 

o Pre pa rat 1on and disseminat ion of reports based on 
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these databases 
o Provision of search and retr 1 eva 1 services . 

/ 

Labs may either use their own resources or help clients find other 
resources 1n or outs Ide of the regl on . 



Table 7. Profiles of Lab Task 3 Policy Improvement Strategies, 
and Policy Stage Emphasized. 



Profile 
Component 



Laboratory 

AEL FWL McREL NCREL NE/I NWREL RBS 



SEDL SEIL 



A. POLICY IMPROVEMENT 
















1 . Inf ormat 1 on 


















a. DB Develop 


X 


X 






X 




X 


X 


b. DB Reports 


X 


X 






X 




X 


X 


c. Search/Retr 


X 


X 








X 


X 




2. Policy KPU 


















a. Research 


















b. Evaluation 


X 










X 




X 


c. Anal/Br1efs 


X X 


X 


X 


X 


X 


X 


X 


X 


d. Development 










X 


X 




X 


e. Tech Assist 


X 


X 




X 


X 


X 






f. Staff Devel 




X 








X 






g. Demonstratl 


on 










X 






B. POLICY STAGE 


A , F A,F,E 


F 


F 


F 


A , F 


I ,E 


F 


F 



LEGEND: 

A. Policy Improvement: X = present 

B. Policy Stage Emphasized: Agenda, Formulation, X m P* lementat i° n • 

Eva 1 uat 1 on 



The specific nature of the databases 1s not described 1n detail 
1n the documentation reviewed. In many cases databases consist of 
materials 1n a file drawer or library collection. I.i some cases a 
formal content analysis of the document has been prepared. 
Retrieval and analysis are often manual but In a few cases files 
have been entered 1n a computer and can be retrieved and analyzed 



ERIC 



111 



CHAPT IV State-Level DMs 



Page IV-14 



electronically. Overlap with Task 1 is suggested by the inclusion 
here of databases on indicators (education, social , demographic, 
economic, e>:c.) and educational issues. 

The study of Information needs of state policymakers by the 
State Policy Consortium found a great Interest 1n the policies and 
reform Initiatives of other states (Cohen 1985a). Four labs 
Indicate that they are tracking tnis Information for states within 
their region ( FWL , NE/I, NWREL, and SEIL). The national 
associations Involved in the State Policy Consortium (see above) 
have collected this kind of data from time to time. However, for 
reasons that are not clear, the state policy coordinators, meeting 
to plan cooperative activities under Task 5, rejected the Idea of 
all nine labs cooperating to produce this database. (The "policy 
Issues matrix" adopted ..is used to keep track of policy issues on 
which the labs themselves are working. See Policy Domains section 
below. Only JrEIL uses 1t to track state reform efforts.) Given the 
strong interest 1n this kind of Information, its relevance to the 
Task 3 responsibilities of all labs, and the continuing interest of 
national organizations, 1t seems unfortunate that it has not been 
possible to organize a national system that would track state 
policies, programs, and reforms on a comprehensive and continuing 
basis . 

In several Instances there are databases on special topics 
related to lab programs . Thus . NWREL has a reglona 1 database of 
Information relevant to the education of at-risk youth 1n the 
Northwest. Other specialized files Include those on service 
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organizations and consultants. 

These databases are used to provide both reports of general 
Interest to decisionmakers throughout the region and to prepare 
targeted reports for given states or even to local districts. 
Thus, McREL reported 1n 1987 that during the past year 1t had 
produced a national , a regional , and two state environmental scans . 

Labs typlca 11 y offer search and retrieval (and sometimes 
synthesis) services to chiefs and other state officials. For 
example, AEL offers to conduct searches and prepare two syntheses 
for er n chief 1n Its region each year. However, a dilemma is 
Inherent 1n these activities. On the one hand, demand for such 
services ,.iay be slow to develop (as AEL found); on the other, too 
much demand will strain lab resources, and labs don't want to 
compete with general infprmatlon centers. 

Policy KPU. Beyond the capacity-building activities of 
Information services are activities or services that are ways of 
doing (or assisting in doing) policy inquiry. Of course this is 
not an entirely clear distinction 1n that doing something 
undoubtedly tends to add to the capacity for doing it. But the 
obverse is not necessarily true; it 1s possible to build capacity 
without actually doing. (e.g., one can learn to do arithmetic 
without using arithmetic to solve problems.) 

Many of the labs have used conceptual frameworks developed by 
William Dunn to distinguish between policy research and policy 
analysis (Dunn 1986). In brief, po 1 1 cy research responds to 
problems Identified by researchers, collects and analyzes new data. 
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operates in a long time frame, and uses complex methods. Pol icy 
anal ysis responds to problems identified by policymakers, and uses 
available Information and simple methods in a short time frame. 

In these terms only FWL conducts policy research, although two 
others conduct pol 1cy eval uat 1on , which can be considered a form of 
policy research. In 1987 , FWL conducted a research study of the 
link between teacher recruitment and retention and a teacher 
Information system in California. It also evaluated the Utah 
Career Ladder System. RBS assisted the District of Columbia Public 
Schools in evaluating a career-focused dropout prevention strategy. 
For FWL and NWREL policy evaluation is a source of contract work 
that operates outside of the OERI scope of work. 

Dunn has elaborated what he cal Is the ADD model of pol icy 
analysis. I.e., types o.f Analysis, Development, and Dissemination 
(Dunn, 1986). This schema would be useful for a more micro 
analysis 1f the necessary detailed data were available and we were 
able to use specific reports or projects as units of analysis. 
Since this 1s not feasible, we are using a more traditional 
Knowledge Production and Utilization (KPU) framework (Mason, 1982), 
but Introduce the Dunn concept of policy analysis as a special type 
of research. 

All of the labs perform policy analysis and/or prepare policy 
briefs. The Issues examined will be dealt with below in the 
section of policy domains. Sometimes the papers are prepared for 
presentation at a policy forum or symposium. In other cases they 
are pr 1nt ed and dist r i buted to sel ected lists of state 
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policymakers. SEIL has chosen to go the sub-contract route for 
doing policy analysis. Agreements have been reached with the 
Educational Policy Center of the University of South Carolina and 
the Center for Policy Studies at Florida State Uni versl ty to 
provide on-demand policy analyses and alternative policy 
configurations for state-level decisionmakers within the region. 
In the Dunn framework, pol 1cy briefs are shorter versions of policy 
Issue papers; labs often distribute policy briefs as newsletter 
Inserts . 

A limited amount of development 1s undertaken. RBS f for 
example, does some Instrument and handbook development as part of 
Its assistance to the Implementation of state school improvement 
programs . 

Another form of service is technical assistance , which may take 
the form of reports designed to assist a specific client, or 
in-person consultation. McREL, NE/I , NWREL, and RBS are most 
Involved 1n this type of work. For example, the New Hampshire 
Assistance Center of NE/I helped the Alliance for Effective Schools 
(a broadly based group promoting school Improvement) to develop 
plans for a school Improvement project based on a model developed 
for special education. Technical assistance is a source of 
additional contract work for some labs. 

Finally, labs provide a limited amount of staff devel opment , 
and 1n some cases develop training materials. McREL, for exam pi e, 
1s conducting a job analysis of the knowledge, skills, and 
competencies needed by state agency staff to prepare for a 



CHAPT IV State-Level DMs 



Page IV-18 



systematic training program 1n strategic planning. 
Pol icy Stage . 

As Indicated earl 1er , the pol 1c y process is a social activity. 
Mitchell and Dunn, among others, have pointed out that the policy 
process goes through several stages and continues to recycle 
(Mitchell, 1979; Dunn, 1986). Their fairly elaborate models 
suggest that different kinds of policy relevant Information are 
most appropriate for each stage of the policy process. 

Table 7 characterizes the Task 3 efforts of the labs in terms 
of the policy stage emphasized. Labs show a marked preference for 
the early stages of the policy process. Pol icy f ormu 1 at ion is the 
modal stage, and several are also active at the agenda sett ing . 
All are Involved 1n the latter to the extent that they share the 
results of their needs. Identification work under Task 1 with 
state- level decisionmakers . 

The clear except 1 on to the overal 1 preference to the earl y 
stages 1s RBS, which appears to operate almost entirely at the 
pol icy implementation and evaluation stages. In the years before 
the current OERI contact Pennsylvania, New Jersey, and Delaware had 
all adopted ambitious stete school Improvement programs. In a 
sense many of the major policy decisions were already made and 
embodied 1n these programs. Consequently RBS chose to focus its 
assistance on the Implementation and evaluation of these programs. 
Pol 1cy Issues and Domains 

The third side of the "strategy cube" 1s the Identification of 
the substantive areas of program work, whether labeled "educational 
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need" or "policy Issue." The strategic planning design for 
laboratories requi res that they sel ect their programs and projects 
through a process of studying the needs and resources of the region 
(see above) . 

Fortunately, as part of work under the state policy theme of 
Task 5 (see below) , a common class if icat ion of pol icy issues has 
been devised. Using this classification, a "policy issues matrix" 
has been established as an on-line database on the Source. This 
database consists of a list of 50 policy issues for which each lab 
indicates the nature of its involvement with the issue, using the 
following code- 

0 Not an organizational theme or policy issue area 

1 An emerging policy issue area or staff interest area 

2 An organizational theme, but not specifically 
identified as a policy issue area 

3 A short-term or one-time state policy issue 

4 An ongoing Task 3 state policy issue 

This database enables labs and others with access to the Source 
to Identify labs that are working in specific policy arenas. It 
also enabl es us to ana 1 yze patterns of lab activity across pol icy 
issues . (Note that it 1s not 1 1mi ted to Task 3 act 1 vi t i es . See 
next page. ) 

Less fortcnately, the classification was not derived 
theoretically and thus consists of a "laundry list." As such, it 
may be useful for searching, but has little analytic value. To 
cope with this problem, a more general and theory-based 
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classification has been adapted that seems to have categories that 
are mutually exclusive and comprehensive. It was developed by 
Mitchell and 1s found 1n the FWL proposal (pp. 11-109-110;. (This 
cl ass If 1cat Ion Is similar to but not Identical to that proposed In 
Mitchell and Encarnatlon, 1983). The 50 policy matrix categories 
are mapped Into the seven Mitchell classes along with one of our 
own . Our adaptat 1on adds a category of pol 1cy Issues defined In 
terms of the class of student Involved (e.g., dropouts, urban) 
rather than an educational Issue per se. 

The codes used to record lab activity present another problem. 
In the present context we are Interested 1n Identifying the policy 
domains of lab Task 3 activity. Tabulating the Incidence of code 4 
would seem to do this, but only 1f we use a narrow definition of 
Task 3 activity, namely that 1t 1s an "ongoing 0 Task 3 state policy 
Issue Interest. In a number of Instances labs have undertaken the 
study of a policy Issue on a one-time basis. Inclusion of code 3, 
"a short-term or one-time state policy Issue," would seem to 
provide the answer. It 1s not clear, however, whether this 1s 
restricted to Task 3 activities or includes work under other tasks. 

Table 8 uses the broad definition of Involvement, indicating 
the number of "policy matrix Issues" each lab 1s involved with at 
either the 3 or 4 level, classified Into the more general policy 
domain categories. (Note that the table entries are numbers of 
Issues addressed, not the Involvement levels.) 

Several patterns stand out. Two-thirds of the policy Issue 
Involvements are found in three domains: curriculum and 
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Table 8. Number of Policy Matrix Issues Addressed at Involvement Level 
3 or 4*, by Laboratory and Policy Domain. 



Policy 
Domain 


















ALL 
LABS 


AEL 


f* lit l 

FWL 


McREL 


NCREL 


NE/I NWREL 


RBS 


SEDL 


SEI L 


Bldg & Facilities 


0 


0 


0 


0 


0 0 


0 


0 


0 


0 


Curr & In:,truct1o 


1 


7 


7 


3 


1 - 2 


0 


2 


1 


24 


Org & Governance 


0 


3 


1 


2 


1 0 


2 


2 


1 


12 


Prog Definition 


0 


1 


2 


2 


0 0 


1 


0 


0 


6 


School Finance 


0 


0 


1 


0 


0 0 


0 


0 


1 


2 


School Personnel 


2 


3 


5 


5 


3 1 


4 


4 


2 


29 


Student Type 


1 


6 


4 


3 


4 2 


1 


0 


0 


21 


Student Testing 


0 


0 


0 


1 


0 0 


1 


0 


0 


2 


Total Issues 


4 


20 


20 


16 


9 5 


9 


8 


5 


96 


Involvement Rank 


9 


1 .5 


1 .5 


3 


4.5 7.5 


4.5 


6 


7.5 


NA 



Source: Ba^ed on Regional Lab Policy Issues Agenda, 2/1/88 
*Level 3 = A short-term or one-time state policy Issue 
Level 4 « An on-going Task 3 state policy Issue 



Instruction, school personnel, and student types. No attention is 
given to "buildings and facilities," and practically none to 
"school finance" and "student testing." Both of the latter are 
perennial state Issues, but evidently labs are not perceived as 
resources for their analysis. 

The labs vary markedly 1n number of issues addressed . Beth FWL 
and McREL address twenty different Issues, FWL 1n five domains and 
McREL in six. At the other end of the spectrum, AEL 1s Involved 1n 
only four Issues 1n three domains, and NWREL and SEIL in five each. 
Only a small association exists between the number of issues 
addressed and budget 1 evel (see also T able 3). 
National Coordination . 

General . Because all labs have Task 3 scopes, state policy is 
a natural area 1n which to expect cross-lab coordination. In 
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addition, important national resources can be accessed by labs. A 
state policy theme was indeed identified under Task 5. Some 
national coordination takes place in connection with this theme, 
and some outside of it. 

Labs are also expected to coordinate their work with research 
centers supported by OERI . In this case there is a Center for 
Policy Research in Education (CPRE) located at Rutgers. While 
severa 1 labs have worked with CPRE to organize regional policy 
forums, it is not clear that they have made use of research, 
conceptualization, or other substantive results coming from the 
Center . 

In organizing state policy forums a number of the labs have 
worked with SEPS (State Education Policy Seminars) established 
jointly in 30 states by the Education Commission of the States 
(ECS) and the Institute for Education Leadership (IEL). 

Labs also part 1c i pate in the nat ion a 1 scei , - by report ing t hei r 
work at the meetings of national professional associations. 
Several labs provided input of state policy data at the request of 
the National Governors' Association. 

Task 5 State Policy Theme . Each lab was required to set aside 
funds under Task 5 for collaborative work among the labs and with 
centers, national organizations with state policy interests, and 
OERI . A theme for collaborative work in state policy was mandated 
1n the RFP by establishing a network of Task 3 project directors. 
This group has met several times 1n the first two years and the 
theme will be continued 1n years three through five . SEIL is the 



CHAPT IV State-Level DMs Page IV-23 

lead lab for this theme, and all labs are participating to some 

degree. Several national organizations have been represented at 

meetings of the network. 

Two objectives were established for this theme: 

o To provide a means for exchanging policy resources and 
products 

o To enhance existing policy support systems with an 

emphasis on increasing the capacity of labs as well as 

state and national agencies and support groups, by 

focusing on six study questions: 

* What are the appropriate ways to do policy analysis? 

* Hew do we assess and enhance capacity for pol icy 
analysis with states? 

* What 1s the nature of data systems to assess and 
support policy development? 

* Can we develop a taxonomy of underlying policy 
mechanisms? 

* How can support groups relate to policymakers? 

* How do we develop new strategies of policy 
1mpl ementat ion? 

From the documentation available it is not possible to quantify 
the amount of collaborative work done under this theme. Scattered 
references are made to one lab providing a speaker at another lab's 
policy foru.-n or making policy analysis papers available. It 1s not 
clear whether any policy analyses were done collaboratively. 

The distinction between doing and capad ty-bu1ld1ng has already 
been Introduced . The principal activities accompl ished , and the 
main thrust of the objectives and "study questions" that guide this 
task theme, are intended to Increase the capacity of the labs 
themselves and of their collaborators and clients to undertake a 
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more systematic and Inquiry-based policy analysis function. 
William Dunn's aforementioned paper (1986) provides a conceptual 
framework for pol 1cy inquiry. A sacond document by Dunn 1n 
collaboration with Richard E. Basom, Jr. and Carl D. Frantz and 
sponsored by NE/I 1s the Educational Policy Development Training 
Handbook . As of the end of the second contract year onl y an 
outline and one prototype section had been completed. 

One other noteworthy accomplishment has been the establishment 
of a "policy Issues matrix" to describe the Involvement of each lab 
(all programs) 1n selecttsd policy Issues. This database was used 
above for the analysis of policy domains. 
Pol 1cy Assistance Configurations 

The fact that 1t was not feasible to use the project or 
activity as the unit of analysis and our method of highlighting the 
"primary" clients, etc. has obscured the full range of each lab's 
activity. It will be useful here to provide k thumbnail 
description of each lab Task 3 program to get a better feel for 
total programs . 

Level of Effort . Before looking at the individual program 
configurations, let us Introduce Table 9, which shows each lab's 
budget for Task 3 1n 1986 and 1987. The RFP specified that each 
lab should devote 10 to 20 percent of it's program effort (I.e., 
Tasks 2-5) to this work. Task 3 effort expected to be secondary to 
the effort for Tasks 2 and 4. 

For t he most part , the effort programmed for Task 3 was within 
the guidelines and on the high side of the range. RBS appears to 
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have committed resources well above the guideline level, but the 
figure may Include some funds for Task 2. 



Table 9, Laboratory Level of Effort for Task 3, FY 1986-87, and Regional 
R&D Organizational Resources. 







Fiscal Year 1986 


Fiscal 


Yrar 1987 




R&D 


Orgs 


Lab 


No. 




Amount 


Task 


3 


Amount Task 3 


No. 


Rank 


States 


Amount 


Per State 


%* 


Amount 


Per State 


%* 


** 




AEL 


4 


132,597 


33, 149 


1 1 


142,662 


35,666 


13 


125 


8 


FWL 


4 


353,400 


88,350 


18 


352, 745 


88, 186 


20 


303 


3 


McREL 


7 


235,838 


33,691 


21 


269,071 


38,4^9 


21 


176 


7 


NCREL 


7 


179,678 


25,668 


16 


89,700 


.12,814 


10 


552 


1 


NE/I 


9 


236,457 


26,273 


19 


269,575 


29,953 


18 


408 


2 


NWREL 


7 


349, 662 


49,952 


13 


275,080 


39, 297 


10 


114 


9 


RBS 


5 


583,375 


116,675 


28 


583,375 


1 16,675 


28 


291 


4 


SEOL 


5 


194,468 


38,8i4 


14 


272, 975 


54,595 


20 


180 


6 


SEIL 


6 


120, 913 


20, 152 


19 


123,024 


20,504 


15 


271 


5 


TOTAL 


54 


2,386,388 


44, 192 


19 


2,378,207 


44,041 


17 


2,420 


NA 



^Percent of program funds. Tasks 2-5 
**Source: Franks! et al 1579 



Just what can be accomplished under this task 1s a function of 
both the absolute amount of money available and the number of 
states to be accommodated. In 1987 the absolute amounts ranged 
from $120,000 (SEIL) to $583,375 (RBS). The number of states or 
other jurisdictions served varied from four (AEL and FWL) ) to nine 
(NE/I). On a per-state basis, the range is from $20,504 (SEIL) to 
$116,675 (RBS). The range for percentage of program budget is much 
smaller, from 11.4% AEL) to 27.8% (RBS). Of course there 1s no 
requirement that the same amount be spent on each state, and some 
activities are region-wide and not state-specific. 

Between 1986 and 1987 SEDL made a significant Increase 1n Its 
Task 3 allocation as its thematic work reached Its final 
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made by NCREL and NWREL. Rationales for these changes are not 
available. 

Regie ia! R&D Resources. One would expect that lab plans for 
state policy asristance would be tempered by the availability of 
resources for policy anal ys 1s within state government and in 
non-governmental organizations. Table 9 contains a rough 
Indication of the distribution of R&D resource organizations by 
region as they existed 1n 1981. (While the numbers may have been 
different 1n 1986-87, the rank order by region was probably about 
the same. ) 

NCREL 1s a new lab still in a developmental stage with a small 
budget and required to serve seven states. NCREL spent only 
$25,668 per state 1n 1986, and even this was reduced to $12,814 in 
1987. On the other hand, the North Central region ranks first in 
the number of R&D organizations in the region. These circumstances 
may account for tne focus of the NCREL Task 3 program on state 
forums that Include outside experts. 

By contrast, NWREL ranks lowest in regional R&D resources and 
has the largest total budget, but 1n allocation of funds to Task 3 
1t ranks third on both total and per-state bases. 

Further comments about 1 evel of effort and R&D resources will 
be made 1n the description of ^ach lab's Task 3 "configuration" 
that follows. 

AEL . AEL 1s in the 1 ower end of the range for both total 
dollars available for Task 3 ($132,597 and $143,662) and in 
percentage of total budget allocated to this task (slightly over 
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nine percent). However, because only four states are in the 
region, the amount per state is close to the average (about $35 r 000 
1n each year ) . 

The Task 3 program, the Policy and Planning Center, is a 
reflection of AEL's organizational design that establishes 
structural interconnections between the lab and major state level 
constituencies. Focusing on Task 3 is potentially misleading, for 
Tasks 2 r 3 r and 4 are all intended to assist different grojps of 
state- level decisionmakers . 

AEL begins by noting that the educational R&D community is 
marginal to the world of educational practice. The lab is viewed 
as a linkage organization designed to assist in bridging the 
"theory-practice gap. M They are aware, however, of the research 
literature failure of decisionmakers to use research-based 
knowledge and the inadequacies of strategies that rely solely on 
information. Their intent is to "maximize long-term involvement by 
state officials in activities that should help them develop a more 
inquiry-based approach to decisionmaking and, at the same time, 
support their efforts at inquiry with easily accessible and 
relevant information" (AEL Proposal 1985, p. 135). 

The program is organized into five activities. The first is 
designed to create a forum of the region's CSSOs . The chiefs, all 
members of the AEL Board, are also constituted as the Council on 
Policy and Planning. As such, they have a dual role, first as a 
committee of the board with special responsi bi 1 i ties for reviewing 
and making recommendat ions on the lab program , and second as a 
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forum for the discussion of regional policy issues. This committee 
meets before the board's quarterly meetings. The first role has 
tended to overshadow the second, and in the second contract year 
only two Task 3 forums were held. A formula has evolved for 
rotating the meetings among the four states and inviting the host 
chief to select the policy topic. He/she also has the option of 
inviting a variety of state-level policymakers to the forum. The 
lab assists by preparing special Information packets and policy 
briefs and commissioning papers by outside experts. 

The second activity 1s an annual symposium of state government 
decisionmakers. The topic for year two was "The Competitive Edge: 
Preparing Youth for Jobs of the Future." The 70 people attending 
Included legislators, state board members, governors' staff, state 
voc ed advisory council members, and business and industry 
representatives. In the future the annual symposium will double as 
one of the chief's forums. 

The third activity is Intended to monitor and report on 
selected educational trends in the region, but it 1s not clear that 
any work was actually accomplished under this activity. During the 
second year a design for accomplishing this task 1n FY 1988-90 was 
produced under sub-contract by the Virginia Policy Analysis Center 
at the University of Virginia. 

The fourth activity is to provide a policy information search, 
retrieval , and synthesis service, including 1 inkages with national 
and regional organizations, for the chiefs and other government 
officials. Eight issues papers have been published. For 1988-90 
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activities three and four have been modified to de-emphasize search 
and retrieval services and to put greater effort Into issuance of 
trend and pol icy Issues papers . 

The fifth activity 1s to prepare "Policy Briefs" as inserts for 
AEL's news letter , The Link . Four such inserts were prepared in the 
second contract year. 

In sum, the AEL program 1s designed to increase the inquiry 
orientation of the chiefs primarily, and other state leaders 
secondarily, by Involving them 1n meetings for the discussion of 
policy Issues of their own choosing and providing information and 
analysis resources to support these activities. 

FWL. Although the Far West proposal describes a system of 
matrix management that has a Task 3 coordinator providing oversight 
for activities found 1n each of four programs, the Task 3 work 
reported for the second year seems to reflect only that labeled 
"State-Level Policy Support" under the Policy Support Services 
Program . 

FWL spends more than $350,000 per year for Task 3. With only 
four states 1n Its region, this amounts to roughly $88,000 per 
state. However, 1t is a heterogeneous region. California 1s a 
very large, resource-rich state, while the other three states have 
small populations and much smaller resources. The program is even 
larger that these figures suggest because the laboratory has 
undertaken several significant non-federal contracts consistent 
with Task 3 objectives. 

The FWL program 1s notable for Its employment of field agents 
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in each state. These agents have a variety of responsibilities: 
they col lect needs-sensing and trend information, update lists of 
key actors, help organize SEP seminars, prepare state-oriented 
Inserts to policy briefs, and prepare white papers (personal 
communication from Paul Hood ) . 

Probably the dominant activity for FWL is preparing policy 
briefs and Issue papers, four of which were prepared 1n 1987. But, 
it also engaged 1n direct consultation with state-level 
policymakers 1n a relatively rare instance of involvement in the 
legislative process: it worked with the California Senate 
Appropriations Committee to develop an omnibus school reform 
measure. The lab also worked with the state directors of planning 
and program evaluation* to explore the establishment of a regional 
database, an Idea that had to be dropped for lack of funds. In the 
area of policy research and evaluation, the lab studied the Utah 
Career Ladder System and analyzed the link between teacher 
recruitment and retention and a California teacher Information 
system. 

McREL. McREL allocates a significant portion of Its budget to 

Task 3 (20-21%), but this amounts to only $33-38,000 per state, on 

the average^ ^Jhe program is noteworthy for its focus on the theory 

and methodology of strategic planning. Their assessment of state 

education systems 1n the region had identified a number of 

deficiencies (Proposal pp. Ill-Task 3, p 1-2): 

o Lack of a comprehensive database to devel op and assess 
school Improvement and restructuring proposa 1 s 

o Lack of a coordinated planning and evaluation mechanism 
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o Lack of policy anal ys 1s 

o Lack of opportunities for state leaders to network and 
to get to know t he per s pec t Ives of ot her 1 eaders 

o Lack of opportunities for training and development 

o Lack of Informational models for restructuring 
educat i on systems 

Further , thel r examination of the context of the region's 
educational systems had revealed a major shift from an industrial 
to an Information-based economy and focused on the restructuring of 
education to adapt to that shift. 

Given this analysis, McREL adopted a strategy for Task 3 that 
emphasized a capad ty-bu1 ldlng approach to help the state education 
systems use strategic planning to adapt to environmental press. 
Five sub-tasks 1n the 1985 proposal had been consolidated into 
three by the second year'. 

First, 1n response to the lack of needed Information organized 
on a state and regional basis, the lab has organized and developed 
a variety of databases, including state files, educational news 
publications, education Influence publications, and statistical 
publications. These databases can be searched through the use of 
key words. The lab publishes and distributes reports based on the 
databases and commissions related policy Issue papers and a 
periodical, Pol 1cy Notes . 

Second, the lab organizes an annual policy conference. One, 
held 1n 1987, Invited 30 state leaders to hear national experts on 
Issues that appeared to be strategic rather than educational Issues 
more narrowly defined, i.e., human resource development, state 
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economic development, school finance, and school/business 
partnershl ps . 

Third, technical assistance 1s provided to state (and local) 
education policymakers. T he lab has assisted North Dakota, Kansas, 
and Nebraska to devel op state st rategl c pi ans and provided planning 
assistance to numerous LEAs. 

NCREL . In Its 1985 proposal (the lab T s first year, but a year 
before the time frame under review) the lab had proposed an 
Information and networking strategy designed to provide useful 
Information to legislatures, SEAs , and local school districts. 
They planned three types of meetings 1n each state each year: 
legislative forums, SEA seminars, and policy Implementation 
Institutes, which would have required 21 meetings a year in Its 
seven-state region . 

However, only $179,678 was allocated to Task 3 1n 1986, or 
$25,668 per state. In 1987 this amount was cut 1n half ($89,700 
total, or $12,814 per state). By 1987 the ambitious schedule of 
meetings originally planned had been reduced to one state policy 
seminar In each state plus a regional policy workshop sponsored 
Jointly with the Center for Policy Research in Education. The 
meetings featured both papers prepared by experts and interaction 
among various stakeholders . Activities Involving database 
development and services, technical assistance, or training do not 
seem to have been used for this task. 

NE/I . Originally the lab defined Its Task 3 program as the 
theme for Public Policy for School Improvement, plus 
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policy-relevant activities under three ether themes, and 
distinguished between regional and state-spec 1f ic projects . In the 
annual and assessment reports 1t is difficult to pull this entire 
set together , and it 1s not clear whether the budget all oca t ion 
refers to the entire set or only the Public Policy Theme. NE/I 
makes an average commitment to Task 3 1n terms of budget percentage 
(17.9% and 18.2%), but given nine jurisdictions, the amount per 
unit 1s near the bottom of the range. Consistent ,*1th its 
decentralized structure, the lab emphasizes a linkage strategy 
using state-level forums, although some regional meetings are 
sponsored • 

NWREL . The Northwest Lab has a relatively large budget, but 
must divide Its attention among seven states and territories. It 
made a significant reduction 1n commitment to Task 3 between 1986 
and 1987 (see Table 9). No rationale for the reduction seems to be 
available. 

Al 1 of the states in the Northwest region are engaged in the 
Implementation of long range plans designed to achieve higher 
levels of excellence. NWREL's Task 3 activities are matrixed 
across the Lab'_ organizational units. The dominant theme of these 
activities seems to be the improvement of database management and 
use through development of specific packages and products, and 
rel a ted technical assistance . The accompl Ishments reported for 
1987 Include development of a special regional database on at-risk 
youth; presentations on specific policy Issues at meetings of 
regional groups; development of methodologies for developing, 
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managing, and using databases and profile Information at different 
decision levels; development of an expanded form of the School 
Improvement Program's School Profiling System and a related 
workshop package; and provision of consultation and technical 
assistance to state agencies, school districts, and other NWREL 
staff. It 1s not clear from the documentation whether the Task 3 
effort 1s targeted at the state level or whether 1t is aim^d at 
assisting local districts use database management for implementing 
state plans. If the latter, 1t would not seem to be consistent 
with the objectives of Task 3. 

RBS. RBS stands out 1n Its heavy commitment to Task 3: over 
$583,000 1n each year, 28% of Its budget, and over $116,000 per 
state. (In some documents 1t appears that the State Leadership 
Assistance Project 1s coterminous with Task 3, while in others it 
seems to Include som" activities from other tasks.) 

The RBS approach to Task 3 1s unique in that 1t establishes a 
specific contractual relationship between the lab and each state. 
For each state the lab assigns a senior staff member to serve as 
state coordinator. In turn each CSSO selects a senior state-level 
decisionmaker to serve as the RBS state liaison person. These two 
develop and modify, as necessary, a written agreement describing 
how RBS will work with state-level decisionmakers and how they will 
participate 1n ^ross-state activities. 

Also unique is the lab's emphasis on the policy implementation 
and evaluation stages. Although the lab often provides planning 
assistance, 1t is generally the planning of Implementation or 
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evaluation activities vis a vis t^e state educational improvement 
plan. However, some assistance 1s provided each state in 
identifying new policy issues and analyzing policy options. 
Although the emphasis 1s clearly on working with each jurisdiction 
separate! y , _1t sponsors at 1 east one «^eg1 ona 1 conference . 

A broad range of types of services 1s provided Including, 1n 
the RBS terminology, planning, planning ass1star.ce, resource book 
revision, training, technical assistance, evaluation. Information, 
research, Instrument and procedure development, workshoo 
development, and conference management. The overall Image 1s one 
of gap-fill 1ng within state school improvement programs. 

SEDL . Between 1986 and 1987 SEDL made a significant increase in 
Its commitment to Task 3 as its themes reached the dissemination 
state (see Table S). The lab's posture 1n Task 3 1s noteworthy for 
Its strong desire to avoid taking sides in political battles 
Involving conflicting Interests. It specifically eschews the role 
of regional problem solver and opts for a more purely informational 
role. Further, Task 3 1s not an Independent program; rather, 1n 
the matrlxed SEDL system. Task 3 activities represent the targeted 
dissemination activities of the several themes around which SEDL's 
programs are organized. The three primary tactics are preparation 
of briefing papers, forums for state-level decisionmakers, and 
one-on-one briefings. (An electronic bulletin board was tested but 
later abandoned.) SEDL emphasizes a state-by-state orientation, 
although some regional meetings are held. 

SEDL's self-assessment found this program to be unnecessarily 
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rlg^d and non-responsive to many emerging needs and issues. 
Consequently, 1n the plan for 1988-90 this profile is substantially 
changed. A new theme called ED-AIDE 1s Identified that takes full 
responsibility for Task 3. It is designed to identify and monitor 
emerging Issues and to anticipate and meet Information needs of 
decision- and policymakers, and to respond with available 
Information to requests that require quick turnaround. It does 
continue to assist the theme programs 1n working with state 
decisionmakers. This reorientation 1s an example of the 
constructive use of self-assessment to make program changes. 

SEIL . SEIL has a relatively smal 1 budget and makes a 
relatively weak commitment to Task 3. Consequently it stands at or 
near the bottom of the range for level of effort. Its $21,377 per 
state (8.5% of budget) 1n 1986 moved up to only $24,369 per state 
(8.8% of budget) 1n 1987. 

Educational reform 1s the dominant theme of the SEIL Task 3 
program. In its first two years the lab established and maintained 
a file of state reform Initiatives and a system for monitoring 
them. It also operated a system of Issue identification through 
environmental scanning of publications and surveys of educational 
personnel. It held a regional policy forum and commissioned some 
policy analyses. SEIL 1s also the lead lab for the Task 5 theme on 
state policy under which an electronic database of policy Issues 
addressed by each lab 1s maintained. This program reflects the 
SEIL style of sub-contracting some program elements. 
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Comments on Task 3, 
A wide variety of 1 inkage strategies were used in assisting 
state-level decisionmakers. While there was some collaboration 
with national organizations, most work was int ra-region . Some labs 
were highly focused on the chiefs and SEAs , while others worked 
with a broader cross-section of state-level persons and 
organizations . 

A wiue range if improvement strategies was used, with all labs 
providing policy analyses and/or briefs. With the exception of 
R3S, all concentrated on the earlier stages of the policy process. 

Of eight broad policy domains, the ones in which most lab work 
was done were curriculum and instruction, organization and 
governance, school personnel, and assisting special types of 
students . 

Ac the risk of over-simplification, the dominant role of each 
lab in its Task 3 program can be characterized as follows: 

o AEL: resource for policy information and analysis for 
CSSOs and SEAs plus organizer of regional symposia. 

o FWL : resource for policy research, evaluation, and 
analysis 

o McREL: builder of SEA and LEA capacity for using 
strategic planning and other methods for the 
restructuring of educat i on 

o NCREL: conference organizer for state policy forums 

o NE/I: convener and facilitator of state policy forums 

o NWREL : resource for devel opment of database management 
systems for use by local systems in implementing state 
school improvement programs 



13: 



CHAPT IV State-Level DMs 



Page IV-38 



o RBS: gap-filler in SEA efforts to implement school 
improvement programs 

o SEDL : dissemination arm of lab thematic programs 
targeted at state government personnel 

o SEIL: monitor of educational reform issues and resource 
for policy analysis 

These represent substantially different approaches to bringing 
an inqu i ry or ier.^at ion to bear on the state policy process . 
Whether they have resulted from careful considerations of strategic 
options, the style preferences of lab personnel, or highly adaptive 
responses to variations in regional needs and resources is 
difficult to say. 

Review of these programs in the future might consider the 

fol lowing factors : 

o The balance between information and policy analysis 
st ra tegies 

o Tne balance between capacity building and doing 
strategies 

o The balance between meeting the needs of SEAs and those 
of other state- 1 evel deci sionmakers 

o The balance between state- by-state act i vi t ies and 
regional approaches 

o The balance between proactive and reactive efforts 

o The balance among different policy stages 

o Whether the program serves state-level decisionmakers 
loca 1 districts 
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CHAPTER V 

PROVIDING R&D BASED SERVICES FOR THE IMPROVEMENT OF EDUCATION 
WITH AND THROUGH EXISTING ORGANIZATIONS: 
TASKS 2 AND 4 

Federal Specifications 

According to the RFP, the core of lab work was 1n Tasks 2 and 
4. The two are treated together here because most labs found It 
difficult to organize their work according to this task structure. 
They preferred Instead to define their programs in terms of some 
priority need and then identify both Task 2 and Task 4 activities 
in relation to that need. Some, however, labs did identify 
separate activities for these two tasks. 

Given the Importance of Tasks 2 and 4, a review of their 
characteristics as defined by the RFP 1s in order. 

Task 2 simply stated that "^bs should "work with and through 
existing organizations to improve schools and classrooms" (p. 21). 
While some provision was made for working directly with schools and 
school districts in exceptional cases, 1t was clear that an 
Indirect strategy for school improvement was mandated as the 
dominant linkage strategy. This became known as the "with and 
through" task. 

The rationale for requiring an 1nd1ract strategy was quite 
str Vjhtforward: lab resources would be very limited and all 
regions had at 1 east some important school improvement 
organizations already 1n existence. Therefore some leverage in 
achieving school improvement could be achieved by collaborating in 
various ways with other organizations in the regional 
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Infrastructure . 

Unfortunately, there were also some countervailing 
considerations, the most important, being the loosely coupled nature 
of the KPU configuration which makes impact at school and classroom 
levels problematic. In addition; the willingness of other 
organizations to work with labs has to be constantly tested and 
negotiated. Finally, even where improvement is achieved, the lab 
contribution might be invisible or difficult to separate from 
others' contributions. 

This dilemma permeates everything that labs do and , indeed , it 
might be said that the central challenge faced by all labs is that 
of developing a linkage strategy to deal adequately with these 
probl ems . 

Three examples of appropriate ac ivities were cited (RFP pp. 
21-22) : 

o Building and strengthening organizational support 
systems for improvement. 

o Engaging in cooperative local improvement initiatives 
with other service Improvement organizations. 

o Broadly disseminating the results of research ... to the 
spectrum of audiences Interested in improving schools 
and classrooms . 

Task 4 required the labs to "work to create research and 
development based resources for school improvement" (p. 25). 
Despite amb-n.jity Introduced by the term "based," the text made 
clear that it was 1 eglt imate for labs to do research and 
development as well as to compile R&D based resources. However, 
the labs were encouraged to undertake only short term research and 
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to leverage their resources by taking part in collaborative 
projects. An issue to be raised (but, alas, not resolved) j 
whether the popular tactic of collecting "promising practices" and 
disseminating them is in any sense an R&D-based tactic rather than 
a craft activity. A related problem is that craft knowledge, which 
has its origin in experience rather tKm disciplined inquiry, seems 
to be accorded the same status by the labs as R&D-based knowledge 
although it is not mentioned specifically in the RFP . 

Note that Task 2 corresponds to the linkage strategy and Task 4 
corresponds to the improvement strategy. According to the formu- 
lation presented here, all lab program activities require both a 
linkage strategy and an improvement strategy. So it is not 
surprising that labs found it difficult to divide their projects 
into Task 2 and Task 4 categories. 

Level of Effort 

The centrality of these two tasks was underscored by the 
guideline that Task 2 was to constitute from 4C to 55 percent of 
the program budget, and Task 4 from 20 to 35 percent, or from 60 to 
80 percent for the combined tasks. (One cannot add the two upper 
limits to 90 percent because Tasks 3 and 5 were expected to take 20 
percent.) The actual allocation of effort for the two tasks in FY 
198^ is shown in Table 10. 

Generally the labs allocated between two-thirds and three- 
quarters of their program budgets to the combined tasks. On 
average, 44 percent went to Task 2 and 27 percent to Task 4, 
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Table 10. Amount and Percent Budgeted for Tasks 2 and 4, FY 1987 



Amount Budgeted Percent of Prog Budget* 

Lab Task 2 Task 4 Total Task 2 Task 4 Total 



AEL 


503,667 


374,182 




877,849 


44 


.7 


33 


.2 


73. 


0 


FWL 


809,220 


421 ,940 


1 , 


231 ,160 


44 


.9 


23 


.4 


68. 


3 


McREL 


544,095 


308,416 




852,51 1 


41 


.9 


23 


.8 


65. 


7 


NCREL 


307,759 


372,243 




680,002 


35 


. 1 


42 


.4 


77. 


5 


NE/I 


724 ,618 


321,033 


1 , 


045,651 


48 


. 3 


21 


.4 


69. 


7 


NWREL 


1 ,401 ,549 


837,028 


2, 


238,577 


50 


.0 


29 


.8 


79. 


8 


RBS 


770 ,058 


513,370 


1 , 


283,428 - 


36 


.7 


24 


.4 


61 . 


1 


SEDL 


630,697 


320,472 




951 , 169 


46 


.2 


23 


.5 


69. 


7 


SEIL 


377,544 


231,817 




609, 361 


46 


.8 


28 


.7 


^5 . 


5 


ALL 


6,069,207 


3,700,501 


9, 


769,708 


44 


.4 


27 


.0 


71 . 


4 



*Tasks 2-5 make up the progr* budget. 



indicating an emphasis on the linkage strategy. NCREL is an 
exception tc this pattern, having allocated 42.4 percent to Task 4 
and only 35.1 percent to Task '2. This emphasis on creating R&D- 
based resources may seem surprising considering that its region has 
more R&D and service improvement organizations than any other 
region (see Table 9, Chapter IV); but the lab tends to create these 
resources by subcontract and other arrangements with outside 
agencies . 

Analytic Approach 
As stated above, every lab program has three dimensions: a 
linkage strategy, an improvement strategy, and a substantive focus. 
Task 2 essential ly defines a linkage strategy (work with and 
through other organizations), but lab programs listed under Task 2 
have the other two dimensions as well. Similarly, Task 4 defines 
an Improvement strategy (knowledge-based resources), but lab Task 4 
programs also have the other two dji/nerps i ons . Many labs structured 
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their programs in terms of substantive themes (our third 
dimension), then matrixed the activities between the other two 
dimensions . 

While determining which of the three dimensions is used as the 
primary Identification of a program may be somewhac arbitrary, it 
1s probably based on which of the three is chosen as the point of 
departure in the planning process. If, for example, working with a 
particular set of partners is the starting point, this linkage 
strategy becomes the program's primary Identification. Planning 
then 1s generally done jointly to select a substantive focus and an 
Improvement strategy and to further refine the mode of linkage. 
If, on the other hand, the lab starts with the need to create 
certain R&D based resources 1n a substantive area, it must devise a 
linkage strategy for bringing these resources to bear on the 
pr obi em. 

Our task would have been greatly simplified if it had been 
possible to se the project as the unit of analysis and to describe 
all lab work in terms of these projects . However , as noted above, 
there are a number of difficulties in doing this. Some lab 
activities are continuing open-ended services that do not 1 end 
themselves to a project structure. Some define their programs 1n 
terms of abstract functions rather than programs and projects 
(e.g., McREL Subtask 2.3: "Network and Broker People and 
Resources"; NCREL Task 4, ''Extend Pool of Available R&D 
Information"). Some labs start with a hierarchical structure of 
programs, projects, and activities, but make only partial use of 
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that structure 1n their annual reports (e.g., FWL). 

The Inability to create an exhaustive database of lab projects 
means that we cannot perform quantitative and comprehensi ve 
analyses of lab work by ta:-k. For example, if we attempted to 
analyze work done with i ntsrmed ia te service agencies under Task 2, 
some of labs would undoubtedly rush forward to say, "Your analysis 
is Incomplete because it doesn't show our work with intermediate 
agencies under Task 4." (In Chapter IV we chose to ignore this 
problem, knowing that not all work with state policymakers was 
found under Task 3.) 

As a result we are eschewing any sophisticated quantitative 
analyses in favor of searching for an understanding of lab 
strategies more qualitatively. The method will vary somewhat from 
section to section. The first analysis is of programs designed 
around a particular need or policy issue (i.e., the educational 
focus 1n our three-part schema). The example chosen is programs 
for improving administrative leadership. First, a set of programs 
addressing this problem will be Identified and described briefly. 
Then Issues such as the following will be addressed: 

o Are there variations 1n how the problem of Improving 
administrative leadership is defined? 

o What know 1 edge base 1s used? 

o What linkage strategies are employed? 

o What Improvement strategies are selected? 

The f ol lowing sect ions will ex^ore new typol ogies of 
Interorganlzatlona" 1 collectivities, research approaches, and 

o 1 4 , 
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devel opmen t /demonstration designs for programs for imp roving 
admin 1st rat 1 ve 1 eadership . 

Improving Administrative Leadershl p 
We begl n with a set of programs that use a substantive focus , 
Improving administrative leadership, as the point of entry. At 
this broad level seven of the nine labs identify a need and define 
one or more programs, projects, or activities.* A listing of the 
relevant work units is found 1n Appendix B. 

There are a number of differences 1n the structure of these 
work units. Some are single-level and others multi-level. In some 
cases the highest ";<?vel •? s a major lab program, 1n others it is a 
single project. 

Profiles of work designs for administrative leadership for the 
seven labs working 1n this area are shown 1n Table 11. Following 
is an examination of each design feature by looking across the 
table horizontally. Those who want to focus on the overall 
strategy for any lab progran/project should look down the columns. 
Problem Definition 

The substantive focus of the work represents one of ou r three 
strategic dimensions . The seven labs represented 1n T a ble 11 have 
undertaken work in the general area we have called "administrative 
leadership." In each case the work has been Justified as emerging 



*McREL pi anned but 1 ater dropped its Administrator Leadership 
Program. It did provide assistance to applicants for Leadership in 
Educational Administration Development (LEAD) Center grants. While 
NCREL's Professional Development Program was focused on teachers, 
1t also has implications for administrators. 
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Table II Rraf 11e of Lab Progr< 



FocuSad on Iwprov Ing Ad»1n 1at rat 1 va LeodaraMp* 



bealan Futuri 
PAOoAAH/rfcOj EC f "name " 



A. ^DOlLtN OEFINtTtON 



I. IMPROVEMENT STRATEGY 
1. Knowledge 
a Roaaarc h 



c Practiced 



2. Info StrvIrM 
a Database develepoent 
a Reports 
c 1 ob nana lot tar 
o Newsletter Inssrts 
a Search 4 rotrloval 



3 Mno«ltdg* Product Ion 
a Roaaarc h 
0 Evaluation 
c Policy studlee 
d Development 



SCHOOL ADMINISTRATION 
AND GOVERNANCE 

A" "pAOSL In Of F INI T ION 
How to bulla' capacity 
for 1t>eu1ry 1n schools 



I IMPROVEMENT strategy 

1 . Knowledge iaao 

a Effactlva schools 
Use of coaputer for 

eenegeeent 
School hOM coaaunl- 

oat 1ona 

0. Ru Ida 11 nee for 
assessing secondary 
raoa d lal softwa. j 

«- Preotleee developed 
1n M. AO*, end MDIfl 

2. tnfo Sorv1«oa 
Topical reports, lab 

ntwiUttir. Inssrts 
1n prof sssoc 
newsletters, aaarch 
and retrieval 



Anouledga Production 
Study group topical 



SUPPORT FOR SCHOOL 
LEVEL LEADERSHIP 



A PROtLEN DEFINITION 
tff acha roaoarch raise* 

atda for principal* 
Isolation of principal 
Failure of cort 1f 1cat 1on 

prog raws to hoop paca 



DEVELOP ADMINISTRATOR 
LEADERSHIP PROORAN 

prorlEn'oef tNt f ION 
S0% of ach edws will 
rot 1ra 1n S yra 
eed » owe thing ooro than 
effective aehoola lit 



R Improvement StratEut 

1 Knowledge Raaa 
a Effactlva aehoola 
Pr oa 1netr loader 



k. Poor Aoalatod 
LeederSMp (PAL) 



Coroor ladder opt 1 on a 



t. Info Servless 



3 Knowledge Production 
Eneelne spedsl 

topic* 1n school 

leadership 
Day auaoor lesdsrsh 

1net Itutee 



LEADERSHIP FOR SCHOOL 
IMPROVEMENT 

AT "pkORLEH " DEF I N I f 16* 
Now to eehe roaoarch 
evelleble for uaa 



■ "IMPROVEMENT STRATEQY 

1 Knowledge Raaa 

1. Serg1ovenn1 'a S forcoi 
nlcal. hueen . education 
al, •yoboHc, a culture 



Info Sorvlcoa 



I Knowledge Product 1 on 
Revtea of eodele A 
thoarioa, concaptuil 
out Una. aodulee 



I. IMPROVEMENT STRATEOY 
Knowledge Raao 
i School lopro v aont 
Effactlva •choola 



0. Identify roaourcea 
for ach 1opr loader ah1 



c. 

I Inf 



o Sorvlcee 



I Knowledge Production 
Study cent off leedereh 
Retesrch on teachora a* 

achool 1epr leader a 
Dav trng aatat '.tie fur 

act. network per»on»al 



RURAL NET MOAN I NO 
FOR LEADERSHIP 

A _ 'pRORL EM" DC F IN I T ION 
Sural laolat ton 
Leederehlp 1a a functle 
pf oultlple rolee 



~ IMPPOVEMFNT STRATEGY 

. Anow ledge Raaa 

■ Ed leederehlp 
Networking 
Collefooratlon 
Effective loader ah1p 
Effective oenagaoent 
Effective org dav 



J Info Sorvlcoa 



I Knowledge Production 
Design training 
eaterlala lor regional 
egency and network 
pei sonnsl 



COOPERATIVE SCHOOL IMPR/ 
ORR EFFECT/ MOMT OCVEL 

•""paorlemoef IN IT ION 
How create sen culture 

of high effort lovele 
How to lop law tech 

end erg leproveeonte 



IMPROVEMENT OF TEACHER 
a A DM PE RFORMAMCC 

A''pR^LEM'DEFlN?fiON 
Now to lepleeent atate 
eenoeted eerf oroenue 
eaeoeaeont ayatooa (How 
do principals lepleeent 
teecher eyeteee? 



R IMPROVEMENT STRATEOY 
Knowledge Raao 
a Effective aehoola 
Seven enabling org 
factors In Clerk 
et al R« 
b Strategic planning 
Or) diagnosis 
Motivation of ateff 
Oeslgn of ataff dav 
Prep 1nts»n 1epr teoea 
Currlculue asnsgeeent 
e Eaeeplery training 



Info eervlcee 



i Knowledge Production 
Identify A rev oaeepler 

progrses A naterlala 
Plati A design training 
eai ar t sis 



H "IMPROVEMENT STRATEOY 
1. Knowledge Raee 



IMPROVING LEADERSHIP 
SKILLS OF SON PRINCIPALS 

A~~PA^t Em'oEFInITION 
How to bu1 IR on one 

network existing 

reaourcee 



e. Existing perf sssess 



t. Info Sorvlcee 
Develop databaae and 
prepara raporta ate 
froa It 



3 Knowledge Production 



R. IMPROVEMENT STRATEOY 
1 Knowledge Raaa 
a Review literature on 
oget A leadership eMU 
of school principals 
Probloe anslyala lit 



b TRO training oodules 
baaed on lit review 



c Coneerclal training 



tnfo Sorvlcoa 



I Know ledge Production 
Identify roaoarch needs 
SyntheaKe roaoarch 
Cowplle guide to trng 
watar 1a la 



(This table is continued on the next page.) 
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ko«4«r«hip* front f pure!) 



too Ign Rootura 
PNOtM/RROJ t cf "name " 

1. Rnowledfo taaa 



B. Ordinary knowlodgi 

e. Predwctei 
t. tatent 

1. Ta bo dwval©e*d 



a. Rracticeoi 
I. Eatant 

I. fo ao dovalepot 

1. tntarorfanltatlonal 
Arrin|«Mntl 

a. Collaboratora 

I. National 

II. national 
a HA 

a. ha 

O IE* 
d SHE 
a. Othar 

0. Cllanta 

I. MMry 

I I . Rant a 
111 Typo 



a lnterp*etet1on af 
•with and throufh' 
1. tlO 'la client 
1.. 100 la client 

a. tlO la collaborate 

b. ttO la apeneer 

C No 110 InvOlvad 



Meatlan 
a. Approach 
a. Servlcee 

I. Info aarvlcaa 

II . loch eeaiet 

III. Staff dav 

a fraln tralnara 
fe Train ataff 
1v Oaoonatrat ion 
w. Natwofk craatlon 



tCHOOi ADMINISTRATION 
•NO GOVERNANCE 

c'lIMKAGE STRMEtY~~ 
I. Rnowlodpa Baa a 
a. OOPA atudy 
Naaara' adoption 
at ©fee 



a. Know loop© of r Of Ion 



d. Eatant i RDa„ Rt. 
SOU 

t tntarorfanltatlonal 
Arranf ooon t a 
a. Collaborator* 

I. Laaa/cantara 

II. ttata prof aaaoc'a 
for ado*, eeh Ma 



k. Cllanta 

I. Ado1n1at ratora 

I I . troad 

III. Individual* 



c. Interpretation of 
'with and throufh' 
Ada* ate. ara cllanta 
prof aaaoca provide 
occaaa 



I. Cooo u n lection 

a. ttata (I raf conf) 

b. I.l.ti 
tnfo aarvlcaa 
Tectvilcel aaalat 

workahopa ate 
School Eaeallanea 
Sltaa (SES) 



SORRORT FOR SCHOOL 
LEVEL LEADERSHIP 



~~~ DEVEL0*~ ADMINISTRATOR 
LEADERSHIP PROGRAM 



: LINKAGE STRATEOV 

I Rnoa I edg a taaa 

a Conf 1 fur at lonal peraR 
Linking afonta. ofoncla 
Organltat lonal c hang a 
Modal a af flald-beecd 
Inoulry 

a 

c Eat rafma Roar Aaala 

Lo*d©rah1p (PAL ) 
TtO • Tralnar of 

trelnere capability 
TSD, Su— r loodorah 

1nat Itutaa 



C L tMHAftC STRATEtY 
I. Rnooladoo taoo 



d. Cere©* 
opt lona 



2 tntarorfanltatlonal 
Arranaaoonta 
i Collaboratora 
1 Laaa/cantara 
11. U Aril. Arlt S*a 
Prlnclpole acadoolaa 
Ad hoc focuaad atudy 
f row pa 



a Cllanta 

I. Prlnclpala 

II. Narrot* 

111 Individual* 



c Intarpratat Ion of 
•with and throufh' 
HI Mad , davalop and 
taat product a with 
SOOa. than holp 
StOa loplooont 



). Coonunlcat 1 on 
a Stato approach 

a i.Si 

Train tralnara 
1A on caraar laddar 
opt lona 

taplaaont auaoor 
loatitutaa in arlt 



Intarpratat ton of 
'with and through' 



LEADERSHIP POP SCHOOL 

improvement 



j. Interorgenlietlonel 
Arrangowonta 
a Collaboratora 



'LINKAGE STfcATEtV 
Mnaailodfo taaa 

POP a atudy 
DfSSt atudy 



B . Eatt Rub directory 
of RSO raaourcaa for 
achool loprovaMnt 
laadorahlp 
TtOi Ooolon network S 
oal laborat iva oodola 
ISO training procaaaai 
d TtDt Action fulda fo 
achool laprovaoont 



Intarorfanltat lonal 
Arranfooont a 
Col leboretore 

I. Laba/cantara 

II. SlOa SEAa. IHEa. 



RURAL HE I WORK |Nt 
POP LEADERSHIP 



'LtNKAbt STRATCOV 
Rnowlodga taaa 



e TtO Procaaaaa to 
proaora raf agencleei 
for tralnlnf achool 



1n 



a Cllanta 

I . ItOe 

II. Nodluo 

III. Prlnclp acadooloa 
by IHEa and ©thore 

c Intarpratat lot* of 
•with ant throufh* 
Prloarlly StOa 



nlcetlen 
a ttata approach 
b. I.S 
Info fr Ot of oatarla 
•oterlele. panpla. 
prograaa * practice 
I ha School loproveoon 
Leederehlp Network 



coopiiAiivi'icMOOt inm/ 

OMR UHC1/ MOMT OffVEL 

linkage 'strategy" 

1. Knowlodga taaa 



2 Intororgenltet lonal 
Arrangaoenta 
a. Collaboratora 
1 

11. Thraa *10© *» be 
••lac tad 



p. Cllanta 
1. SlOa 
1 i , troed 
111 SEAa and IHEa 

c . Intarpratat Ion of 
•with and through" 
SlOa ere cllanta 
Producta taat ad * 
tralnlnf aupervleed 
In achool » 



J. Cooounlcatlon 
a State approach 
b 1.4.1 
i Oooot Su par v plan S laple 

trng of ech net paraonn 
Train aupport egencV 

pereonnel 
fulld caPac of aupport 

•fanclaa to aerve 

rural dletrlcte 



MrROvBNfNr df'ieacmek 
ft aqn pf aroauAMCi 



C~~L iHKAf E STRATEGY 
Knowledge taaa 



c TSD Acadawy on 

planned change 
TtO Leedarahlp aoolnar 



Intarorganltat lonal 
arraogaoonta 
Col laboratora 



a Cllanta 
1 SOOa • Sfiaa * 
central offlcoe of 
larga LEAa 



c Interpretetlon of 
•with end throufh' 
LEA teene * networhe 
of than, tlf 100a 
mm 110a 



) Cooounlcatlon 
a State approaeh 
b J.S 
Aceaeay of planned cl 
Laederhahlp eooinere 



'in'ROving' i i*pf asHi a 
Skills Of *c* principal 



LIHKAiiE ITRAIL^t 
Knowladg© teee 
■. Opan ayataoa theory 
Cont infancy thoory 



e PAL (aaa END 
Concarne beeod Adopt 1 
Lab tralnlnf oodulea* 



d Rog lonal pract1cee> 
Parforoanca aeeeeeoont 

■yatooa 
j. Interorg enUet 1onel 
Arrangement a 
i col leboretore 
1 . 
11 . 



b. Cllanta 
1. State-level 
doclelonoahere 



c Intarpretet 1on of 
•with and through 
So*© mm Taak 3 tiee t 
ttete-lavel OMa ft thoa 
reap for training 



I. Cooounlcatlon 
a. Stato approech 
a 1. Olaeeolnete 
pract Icaa 



1 Intarorfenlietlonel 
Arranfooonta 
a Collaboratora 
1 . Labe/eentere 
11 SEAa. IHEa. ISAa. 
diet level principal* 
training prograaa 
Uae roglonel conawltant 
ft nr Sc1 ft lech Reaaar 
Center 

b CHente 

Network of dlrectore © 
prlndpelehlp acedeola 
plua LEAD centere 

c Intarpratat ion of 
•with end through' 
SlOa ara cHente 



I 
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a i.i. i. I 

Out Info * notarial 
IA to I € AC comai b 
SO throufh network 
Fora network 
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from the need assessment process, but that does not mean that the 
labs are working on identical problems. Under this broad rubric is 
considerable room for variation in how the problem is defined at 
more specif ic 1 evel s . 

There is also tremendous variation in the amount of detail 
available on problem definition (as well as other program design 
dimensions). Some labs have prepared special papers (not examined 
1n this review), while others provide only a paragraph or two. 

General 1 y , there are sever a 1 broad themes : 

o Local administrators have a great res pons i bi 1 i ty for 
implementing changes resulting from the reform movement 
and reform initiatives of the states. 

o There are important bodies of research on school 
improvement, instructional leadership, etc., that 
administrators need to become f ami liar with. 

o Sc hool s and t hei r 1 eaders need to devel op a culture of 
inqui ry to use knowl edge- based resources in their 
attempts to improve- schools. 

o School administrators are isolated, and new structures 
are needed that put them in communication with their 
peers and enabl e them to access knowl edge -based 
resources for school improvement . 

o It is a mistake to focus exclusively on the 
principal f s rolo; leadership is provided by a number of 
roles at the school and central of f ice 1 evel s . 

These are not mutually exclusive problem definitions and might 
easily be strung together as one complex rationale. However, the 
tendency is to stress one or a few of these rationales. 

Improvement Strategy 

The use of knowledge-based strategies is a given for labs. 
(Money and regulation are alternatives available to governmental 
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entitles. See Mason 1982.) The components of knowl edge-based 
improvement strategies are the knowledge base, information 
services , and kn owl edge product i on . 

Knowledge Base. There are distinct differences in the 
know 1 edge bases used for the Improvement strategy . (Note that 
knowledge bases are also used for the linkage strategy. See 
below.) Here the focus is on the nature of the knowledge be 
transferred. First is a choice to be made among research 
knowledge, know! edge 1n the form of devel oped products , and that 
represented by existing practices or some other form of craft 
knowl edge . 

The main bodies of research knowledge mentioned are those on 
effective schools and leadership. Some labs provide a discussion 
of one or more of these knowledge bases; some list a number of the 
generalizations that have emerged, and others merely name the 
knowledge base used. At the activity level some specific bodies of 
knowledge come into play (e.g., school -home-communications at AEL). 

A distinction can be made foi products between those that are 
extant and those that are to be developed by the lab. In the 
former category, FWL proposed to use (and further refine) Peer 
Assisted Leadership (PAL), a product developed in prior years. In 
the latter category, AEL proposed to develop a document, Guidelines 
for Assessing Secondary Remedial Software. NWREL planned to 
develop training materials for use with school network personnel. 

Examples of practi ces to be collected and reviewed include 
"exemplary programs" (RBS), "existing performance assessment 
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systems" (SEDL) , and "career ladder options" ( FWL) . 

Information Services . It may seem strange to see the category 
"Information services" under improvement strategy since this would 
seem to be Inherently a 1 Inkage strategy. However , the front end 
activities Involved in Information services are appropriate, i .e. 
database devel opment and the pre pa rati on (in contrast to the 
dissemination ) of reports , news letters, inserts, briefs, etc . based 
on knowledge bases . Database development may be difficult to 
distinguish from knowledge bases used (see above). The term is 
used to refer to organized and manipulable knowledge bases. 

AEL provides a # range of information services, Including writing 
topical reports, preparing topical pieces and Inserts for the lab 
newsletter, preparing Inserts for the newsletters of administration 
professional associations, and search and retrieval services for 
regional administrators. (These activities correspond to 
development 1n the Dunn ADD model for policy analysis (Dunn 1987). 
SEDL compiled a database on performance assess. <ient systems and 
prepared reports based on 1t. It is noteworthy that only two of 
the seven labs chose to use information service strategies. 

Knowledge Production . Knowledge production can take the form o 
research , evaluation . pol i cy studies , or development . SE I L 
expected its activities 1n the first two years would lead to the 
Identifying needs for future research. NE/I proposed two specific 
studies , one of central office leadership , and another of teachers 
as school Improvement leaders. .FWL and NWREL organized and 
developed training materials for institutes and NE/I prepared for 
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publication a Directory to R&D Resources for School Improvement 
Leadership. 

It is often difficult to distinguish between the improvement 
and linkage aspects of a given effort. For example, 1n organizing 
a leadership institute, the instructional content constitutes the 
Improvement strategy while the; ufganization of the institute (e.g., 
whom to Invite from what organizations, whether to organize 't on a 
state or regional basis, whether to use large group instruction or 
org 1ze subgroups by roles or some other criterion) constitutes 
the linkage strategy. 
Linkage Strategy 

The three components of the linkage strategy are knowledge base 
1nterorgan1zat 1onal arrangements, and communication mod:. 

Know! edge Base . The krowledge base for tne linkage strategy 
consists of bodies of kno*ladge concerning the most effective means 
of knowledge dissemination, Implementation, and utilization. It 
can take the forms of research or ordinary knowledge, products, or 
practices/craft knowl edge . 

Documentation of the linkage knowledge base tended to be thin 
or absent in the material describing these programs. This area was 
covered in the original proposals under "lessons of research and 
experience in terms of implications for effective strategies to 
address [each task]" (RFP p. 36). Because it probably applies to 
all programs, it 1s not repeated for each. The citations in Table 
11 are taken from both the proposal Introductory material and 
program/project Documentation . 
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Research cited as the basis for linkage strategies included the 
General Purpose Disseminat ion Assistance (GPDA) Study ( Louis et a 1 , 
1984), the Dissemination Efforts Supporting School Improvement 
(DESSI) Study (Crandall et al, 1983), Rogers' conceptualization of 
adoption stages (Rogers, 1962), open systems theory, and others. 
The only example of ord1 nary know! edge cited in association with 
this program was knowledge of the region (AEL), although all labs 
provided this kind of knowledge 1n other sections. 

For linkage knowledge products , there 1s again the distinction 
between existing products and those to be developed. FWL proposed 
to use and further refine Its previously developed Peer Assisted 
Leadership (PAL) product. (PAL was also mentioned above as part of 
the Improvement strategy knowledge based, assuming that PAL 
contains both substantive knowledge and a linkage strategy.) It 
further proposed to develop a trainer of trainers capability and to 
der^g^ summer leadership institutes. NE/I took the route of 
compiling a directory of extant school improvement leadership 
resources. NWREL proposed to design network and collaboration 
models 1n one activity and develop training processes for training 
school network personnel in another. SEIL proposed to use products 
developed by two other labs and centers (FWL's PAL ^and the former 
Texas Center's Concerns Based Adoption Model) and develop its own 
training modules . 

Similarly, practices can be existing or planned. AEL noted 
that it had a good deal of experience with practices growing out of 
Its work with several N I E-s ponsor ed dissemination programs: RDx 
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(R&D Exchange), R&D Interpretation Service (RDIS) and RS (Regional 
Services), a claim other older labs might make as well. NE/I 
planned to devel op an Act i on Gui de for School Improvement (probably 
a blend of improvement and 1 inkage knowledge) . 

Interorganlzatlonal Arrangements . Possibilities for nationa l 
col laboration lie chiefly in the improvement strategy , mainly in 
developing the various knowledge bases. Linkage is inherently a 
regional function at the service delivery end, although there are 
also possi bilities of national col laboration in terms of reviewing 
the knowledge base for a linkage strategy and developing linkage 
♦ models. Several labs list other "labs and centers as national 
collaborators, but few details are provided. Regional 
col 1 aborators include SEAs , professional associations, institutions 
of higher education, principals' academies (with varied 
sponsorship), and study groups. Clients included both service 
improvement organizations (SIOs) and service delivery organizations 
(SDOs) . 

Each lab's configuration of relationships with collaborators 
and clients is summarized by categorizing its interpretation of the 
"with and through" strategy . Focusing on the SIO as the sole 
client represents a " pure" indirect strategy. The approach of NE/I 
and NWREL is indirect in that they treat SIOs as primary clients 
and assist them in implementing programs with SDO personnel . R8S 
works with SEAs, ISAs, and LEAs . Its work with large metropolitan 
school districts can be considered "with and through" in that 
central offices of large districts function much like SIOs. FWL 
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also wcrks with the central office of a major urban district, not 
only as a client but also as a collaborator, in developing and 
testing products for use in its own district. On the other hand, 
AEL sees SDO administrators as its clients and state professional 
associations as its channels of access to them (i.e., sponsor). 

Communication Modes . Virtual ly all labs organize their primary 
activities by state, although they held some regional conferences. 
The dominant service strategy is staff development. Variation 
occurs 1n whether new networks are organized or existing structures 
(e.g., prlncipalship academies) are used; whether new linkage 
training is designed for SIO personnel; and whether the lab 
provides Information or training to SDO personnel or assists or 
supervises SIO personnel in providing such assistance. 
Conclusion to Improving Administrative Leadership 

Our method has necessar i 1 y broken lab programs into pieces . 
Let us try for a more holistic view. These programs are all 
focused on Improving knowledge and skills of a particular role 
group: mostly building administrators, but including central office 
staff and others 1n many cases. There appear to be two main 
approaches to the Improvement strategy. One is to start with 
research literature and to develop training packages. The other is 
to col lect , review , and package part icul ar admin i strati ve 
practices. The pr inc i pa 1 shi p academy, either pre-existing or newly 
formed, 1s the most popular vehicle for linkage. Most labs use an 
Indirect strategy, training the staffs of SIO agencies, and 
sometimes working with them to implement training programs. Most 
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are organized by state rather than by region. It does not appear 
to be an area of significant national collaboration. 

While use of inservice training is traditional, two aspects of 
most programs incorporate the findings of recent research . One is 
the recognition that the need for leadership resides not in a 
single role but a combination of roles at both the building and 
central office levels. The othr.r is that single-shot training is 
only marginally effective and that more lasting effects can be 
obtained by involving trainees in a continuing network and 
providing various support services. 

We now turn to a more fine-grained analysis of the 
interorganizat ional arrangements used as part of the linkage 
strategy in Task 2 and Task 4 programs. 

Linkage Strategy: Types of Interorganizat Ional Arrangements 
Conceptual Approach 

In moving to a focus on the linkage strategy, it will be 
necessary to shift methodology. In previous analyses we have 
examined three aspects of the linkage strategy: the knowledge base, 
communication mode, and Interorganizat ional arrangements . However , 
the "with and through" strategy was perhaps the most salient 
Innovation in the lab RFP design, and Interorgan i zat 1 ona 1 
arrangements made by labs to implement this design feature warrant 
a more fine-grained analysis. Understanding the nature and 
variations of this Indirect strategy 1s crucial to designing the 
next round of contract awards. 
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The field of interorganizat i on theory has been developing for a 
number of years as an offshoot of open-systems theory (Negandhi, 
1975). It has been studied in health management among other fields 
outside of education (S. Mason, 1979). in education the present 
author prepared a multi-award RFP Issued by NIE for studies of 
interorganlzat ional arrangements that resulted in several 
ground-breaking reports (Huberman et al , 1981; Yin and Gwaltney, 
1981 ) . 

The interorganlzat iona 1 collectivity (IC) has been proposed as 
the basic unit of analysis (Van De Ven et al , 1975). Note that not 
all relationships constitute collectivities, e.g., a lab and an 
organization receiving one-way communications. The relational 
properties of IC's have been examined from two perspectives: (1) 
the dimensions' of Interaction or exchange between organizations 
(e.g., formalization, intensity, reciprocity, and standardization); 
and (2) the mechanisms for coordination between organizations 
(e.g., mediated or unmedlated; domain consensus, complementarity of 
resources, homogeneity of structure, mutual awareness, and 
stability between the organizations in the exchange relationship). 

Unfortunately, documentation available for this study does not 
permit this level of analysis. However, it would be helpful if we 
could get a clearer description of the different kinds of ICs 
developed by the labs. Four kinds , can be differentiated by 
observing two different factors. First, does the collectivity have 
a single or multi-purpose? Second, 1s the collectivity limited to 
two organizations (i.e., a bi-lateral arrangement), or are more 
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than two organizations Involved? The combination of these factors 
yields four kinds of ICs. (See Figure 7). Of course our meanings 
for these terms do not necessarily correspond to those used in the 
labs* reports. The reports being examined make many references to 
"networks" 1n particular, but this term seems to have different 
referents. The Intent here 1s to use four terms and give them more 
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Figure 6. Types of Interorgan1zat1onal Collectivities 



prec ise meanings . 

We can take the classification system several steps further. 
First, we need to note the lab's relationship to the IC in 
question. In some cases the lab 1s simply a member of the IC. In 
other cases it 1s not a member but stands 1n the relationship of 
service provider to a client IC. Adding the membership factor to 
the four types of IC yields eight types of IC configurations, as 
shown 1n Figure 7. 

Previously we had noted that the educational infrastructure is 
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Figure 7 . Types of Lab/In ter organ 1 zat 1onal Col lectlvlty 
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made up of three kinds of organizations, KPOs, SIOs , and SDOs (see 
1 1-1.) For present purposes KPOs and SIOs are combined and labeled 
SIO. 

Analysis of Lab Programs 

In Table 12 the four kinds of ICs appear as the column heads. 
Not all relationships., constitute ICs. It 1? very difficult to 
define the boundary between interorganizat ional arrangements that 
constitute collectivities jnd those that are too evanescent to bf. 
counted, but activities involving one-way communication from the 
lab to some organization are not shown. However, the stub of the 
table shows the types of organizations the lab is related to and 
Its role relation to them. 
*■ A number of lab programs, projects and activities are 

classified 1n the cells of this table. No claim 1s made that this 
tab! e 1s complete or comprehensive . Many projects or other work 
units are no doubt missing because it was not possible to work out 
a system for cl assi f y1ng everyt hing . The goal is to examine 
projects 1n several dlfferef cells to gain a clearer understanding 

° 15 5 



CHAPTER V TASKS 2 ft • 



CHAPTER V TASKS 2 ft 4 



Table 13. Llnkege Strategy t Cleeelf 1oet1en of Selected TmK 
end Leb Kola. 



2 and 4 Lab Prograee/Projecte by Typa of Interorganlzatlonal Collectivity. Typo of Organisation. 



nMp 
Typa of Organisation 



A* LAS 



IS A NENSER 
Sarvloa 
Itprov mn t 
Organ 1 sa 1 1 on ( a > 
(tlOa) 



Lab/aantor aonforonoo aponaorahlpot 

C PRE /NCREL, CPRE/SEDL, CPRE/SEIL 

CVE/AEL. NREA/McRel .etc . 
RSSt Applied Roe/Policy Stud i eat Md 

High Sch trad Requirement a Study 
RSSt Applied Ree/Iapr Studaat Pa ft 

Md atudloa of oandatad elnlejua 

eoopotoncy tootlnf 
RBS ft CIRCEt Effect a of change a 

1n aeeeeeoent policy 
FHL I • 1 i Suppt for Sag Taaa w/CSU 
FHL 3.3 i Preparing taaa for work 

w/d1veree etud pope. w/CSU 
NM 4.1ft The Automated Workplace t 

Jt dev m/ oorporete pertnere 
SEIL end McREL: contract e w1t» IHEe 

ft other egenoloe for ROftE 



2. Service Delivery 
Orgenlsetlon(e) 
(SOOe) 



Mlaed SlOa 
end SOOa 



"LAl'It'SPOHSOi/ 
SERVICE PROVIDER 
1. Service 

leprovaaant 

Orgenlsetlon(e) 

(•10.) 



Sing 1 a-Pur poee 
S1-Letere1 
PARTNERSHIP 



LAS PARTNERSHIP 



FHL 3. St Promote dletrlct opt lone 
for ach 1cpr w/Clerk Co HV 

HcRELt School Audit Experleoent 
with Shoahone. HV 



"CLIENT "PART MERSH I P 



RSSt Applied Rea/Pollcy Studlee. 

4 pol etudlea done for Pa loglel 
SEIL contract* with IHEe 
AEL t Cleeerooe Inetr Prog: provide 

TA to NEA'e Meetery 1n Learning 
McREL t Reeource eupport for 

ISAe 1n NE. KS. end CO 



•Ingle-Pur poee 
Nultl-leteral 
CONSORTIUM 



LAS CONSORTIUM 



Lab Taek S ti 
CSAPt C o— unlcetlon Sorvloae 

Aaaletence Prog (Leb network 

eotlvlty done by CeOAR) 
RSS ft Concert 1ue on Ed Policy 

Studlee. Indiana U. 
NH 2. It Info Ctr on Application 

of Technology t RICE datebeee of 

eoureewara ovele fro* "network" 

of SEAa 

NH 2. ft Col loglel Tee Trng (w/ 

SEAa laAm. ft IHEe 
NH 4. 4 i Technological Literacy 

for All i elaed oeetlng with aubj 

oetter end tee organ 1zet1 one 
FHL I.St Teaching for Scl Llterecyjt 

w/CSU ft Kern Co CA 



RSSt Study of 10 High 3c ha with 
Mid Atlentlc Me to Council 

NCREL S.3t Curr Doelgn In Tee Ed 
eet e concert 1ue to e pone or the 
etudy of Induction proge 



SEIL: SEED (Softwere Eval Exch) 

with SEAe end LEAe 
FHL 1.3: Sch Oropout Taek F. 

Phoenix, w/ etete leg. bue. ate. 
FHL 2. It Outcome Saeed Ed «*/ 

UT SEA ft ISAe 



CLIENT CONSORTIUM 



SEILt "Network* of alto direct ore 

of SE Hr1t1ng Project a (SEHP) 
SEOL: Theeea A. S. C (or 

ere theee broedceetlng rel'e?) 
NCREL t Video conf w NON feclllteta 
NCREL t Technology Info Exchenge 
FHL 3.2t Provide TA for lepleeent 

prof dev ctr net f pup echo NV 
AEL t Oeelgn ft teet Inetr etretegy 

for uee 1n teacher ed 



Mult 1-Purpoae 
•1-Lateral 

COLLABORATIVE 



Mul t 1-Purpoee 
Multi-Let ere 1 
NETWORK 



LAB COLLABORATIVE 



RBS/SEA agreeeenta (Taek 3?) 
FHL/CSU (CA Stete Un1v) egreeeent 



iltt Teaenar~etu3y~groupa~ 
kELt Ada etudy group a 



CLIENT COLLABORATIVE 



" L A I "ii E f HO RK " 



SEIL. NCREL (end other a) Stete 

Advlaory Coewltteee 
NCREu: tete L1a1eon Coeolttee 
Leb portion of Urben Ed Network 

AEL. McREL. NCREL 



CLIENT NETWORK 



NH 2. lit Regional Indian Ed Net 

(Stete Indian Ed Aeeoce) 
NE/Ii Leederehlp for Scho lopr Net 
RBS: Coop Sch lopr Networke: 
o MO Council of Aaelot Supte 
o Labor-Mgmt Cooperation Network 
o Mid Atl Metro Council (• LEAe ) 



ft 

ID 

(This table is continued on the next page.) § 

< 
i 

-A 



to ' 

ERIC 



CHAPTER V TASKS 2 4 4 



CHAPTER V TASKS 2 4 4 



Table 12. Linkage Strategy* Cleeelf 1cet 1on of Selected TasK 
and Lab tola. (continue**) 



2 and 4 Lab Progreee/Projeota by Type of lnterorgen1set1one1 Collectivity. Typa of Organisation. 



Irani p 
Typa of Organisation 



Sarvlca 

Organ 1s 
(fa Out) 



Dal 1 vary 
•t1on(») 



Nixed SlOa 
and SDOa 



S1 ng 1 a - Pu r poa a 
•1-Laterel 
PARTNERSHIP 



~ClI1n¥~ PARTNERSHIP 



•1 no la- Pur pose 
Multl -Lateral 
CONSORTIUM 



CLIENT CONSORTIUM 



RtSt Appllad Raa/Po11oy Studies* 
strategic planning stud :•■ in 

LCAa and looal achools 

NH 2. It Training Ctr tor c isssra 
Aaaasst trng taaa. trelnera, pra 
oollab « SEAa. ISAa 4 iHEa 

NH 2. lit Pac Local Capacity lldg 

NH 2. Its Problem Clarification/ 
Need IDi (Sarviaa %• ind LlAi) 

NH 4. IS i Voc ad pilot taat altaa 

FHL 1.2a Se1f-01rected Laarnlng In 
So1 w/Heehoe Co NV, Nor ke hop Hay 
proj w/Dav1e Co UT 

FHL 1.4 1 CSU training of taaa In L< I 
FHL 2.2t Uaa of Taoht Plan 4 Sarv 

w/CA SEA i raa 4 plan w/NV SEA 
AEL s CI scare Inatr Progi on daaand 
TA to achoola 

"NCREL"3laaaa)1nanon"partnar8 
(algnad agraaaanta ) 



SEOLt Tnw* 0 Oaae altaat Lab 4 

LEA-S10 pair 
RSSi Flf>Vj tasta of apaclal pops 

school l^oroveeent acaa l 
NH ?.10. Et»*er1ve Prictlcee In 

Indian Educatlcn 
NE/Ii Taa Oavs To* Chang a Nat 

for eeeler aupta 
FHL 2. Si Raaeurca Support fo CA 

County Offices of Ed 
NcRELi Doeo altaa 1n KS 4 CO 



IcRfLi with Oenver Lfc*. cnerry 
Creek Schools. Valdaa and H111c 
Craak 



SE0LT~»tsta"Taval"a»at1ngs" 
NE/It Ed-Work Partnarahlpa 



Hultl-Purposs 

S1 -Lateral 
COLLASORAT I VE 



CLIENT COLLABORATIVE 



MH~2TTT~R^raT~fiotwork1ngT Side 
( aostly iHE'a) aach run rural 
not with NH hulpx d1ff foci . 



" Multi-Put >oss 
Multl-Lsterel 
NETWORK 

*5LlESf"SEfHORK 



NH 2.12: Ed Natworka for Sch lepri 

El Nat, M1dd1a/Jr Nat/ Sr HI Nat 
HcREL: LEAD Canter Network 
McRELs Rural Network 
AELt Claaarw Inatr 4 Gov/Org 

Progs i Study Groups 4 Rag Conf 
p 03 i Coop Sch lepr Network i 
3 Rural 3che Exchange (300* dlata) 



Lab w §T67s56T"Urban'Ed"Network 

AEL. McREL. NCRELj aupta 4 prs 
HcREL Staff Dev Network 



01 
0 



< 
I 



9 

ERIC 



CHAPTER V TASKS 2 & 4 p age V-23 

of what these different linkage arrangements mean in practice. 

Despite this disclaimer I will hazard two general izations . 
First, the collaborative (mul t1-purpose, bi-lateral) seems to be 
the least popular form of IC. The agreements that RBS has drawn up 
with each SEA 1n its region (which apply primarily to Task 3) fall 
1n this category. Similarly, the relationship between FWL and the 
California State University (CSU) covers a range of activities. In 
fact the two organizations overlap in that the Southern Service 
Center operated by FWL is shown as a unit in the lab table of 
organization. At the bottom of the chart, the NWREL Rural 
Networking Program 1s really a three-level arrangement and 
therefore classified as "mixed." The lab provides support services 
for a number of SIOs (presumably separately), which in turn provide 
services to a network' of rural schools. Perhaps the lab/SIO 
relationships would be better categorized as partnerships and SIO 
relationships as client networks. 

The second generalization cautiously advanced is that there are 
far mere single-purpose arrangements (partnerships and consortia) 
than multi-purpose (col laboratl ves and networks). This seems to 
point to the desirability of flexible arrangements for responding 
to shifting needs and opportunities. 

iCs with Lab Member . The top half of the table contains ICs in 
which a lab 1s a member of the collectivity. It is undertaking an 
activity jointly with one or a series of organize t ions on a more or 
less equal footing . Variation occurs in terms of whether the other 
organizations are SIOs, SDOs or mixed, whether the activity has a 
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single or multiple purpose, and whether the relationship is 
bi-lateral or multi-lateral. 

There were a number of cases of partnerships between labs and 
SIOs. A frequent activity was Joint sponsorship of a conference 
with a research center. R8S reports working on research studies 
Jointly with the Maryland and Pennsylvania SEAs. It had another 
partnership with the Center for Instructional Research and 
Curriculum Evaluation (CIRCE) at the University of Illinois. The 
FWL/CSU relationship, already noted at the Institutional level, 
shows up again as partnerships for specific projects. 

Although business corporations are not, strictly speaking, 
SIOs, one program Involving corporations is placed in the SIO 
partnership cell. NWREL worked with several such corporations 1n 
Its Automated Workplace- program to get a more accurate idea of the 
work environments that students would be moving Into. 

There was one case of a partnership with an SDO : FWL f s work 
with Clar'- County, NV, on the project, "Promote District Options 
for School Improvement." 

The consortium may be the most frequent form of lab membership 
in an IC. Individual themes under Task 5 would be the clearest 
exam pi e : three or more co-equa 1 organ 1 zat ions working together on 
one topical theme. Similarly, joint lab sponsorship of CSAP, 
(Communication Services Assistance Project) at CeDAR is a cross-lab 
activity, operated in this case by a third-party SIO. 

NWREL has three projects Involving multiple SIOs. In one, SEAs 
provide software evaluations for a database operated by the 
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Information Center on Application of Technology. In another, a 
meeting focused on Technological Literacy for All was attended by 
both subject matter and teacher organizations . And in a thi rd , it 
worked with SEAs , ISAs, and IHEs on the Colleglal Teacher Training 
Project. 

Labs are also Involved in consortia with SDOs . RBS made a 
study of 10 high schools working jointly with the Mid Atlantic 
Metropol itan Counci 1 (composed of SDO administrators) . NCREL 
established a consortium to sponsor a study of Induction programs. 

In the "mixed consortia" category , SEI L operates the Software 
Evaluation Exchange (SEED), apparently similar to the NWREL 
project, but involving both SEAs and LEAs as sources of 
evaluations. FWL worked with a Dropout Task For . j 1n Phoenix, AZ , 
with both the LEA and ..community representatives. It also operated 
an Outcome-Based Education project ,i Utah 1n conjunction with both 
the Utah SEA and state ISAs. 

Continuing across the top half of the table, we have already 
noted the paucity of lab col labor at 1ves . Lab networks are also 
relatively scarce. The Urban Education Network is somewhat 
difficult to classify. It is operated by thr e labs, so there is a 
lab network at that level. But as a whole it is bettor classified 
in the lower right box as a "client network," since the labs 
provide support for SDO members of the network. 

CI lent ICs . Although not Task 2/4 projects, state advisory 
committees and liaison committees operated by a number of labs 
might be considered cl lent networks of either the SIO or mixed 
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variety. RBS works with four networks, three at the SIO level and 
one at the SDO level. In the former group are the Maryland Council 
of Assistant Superintendents and the Mid- At 1 ant ic Met ropol i tan 
Council (the five 1 argest LEAs in the region) , being counted as SIO 
personnel here because central offices of large districts function 
like SIOs; end the Labor-Management Cooperation Network. In the 
latter category is the Rural Schools Exchange. McREL operates a 
rural network and a l £AD Center network. 

Improvr - ient Strategy * RD&D Approaches 
We would also like to develop a more fine-grained analysis of 
some of the Improvement strategy categories. 
Research Approach 

In previous analyses we have used a breakdown of the research 
approach *nto research evaluation . pol icy analysis , and pi anninq . 
In Table 13 these categories appear in the stub. For each of 
these, a further differentiation can be made in terms of the 
primary source of data, i.e., primary data , secondary anal ysis , and 
interaction documentation . These categories appear as column heads 
in the table. Tne first two of these are straightforward. The 
third, "Interaction documentation , " needs some explanation. Labs 
hold many meetings with other groups and individuals. Frequently 
these are "working meetings " rather than listening to speakers . 
When people share ideas and interact at these meetings , a 
documentation of what occurs is often made thot constitutes an 
Intel lectual product termed " interaction documentation . " 
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Table 13. Iiprovmnt Strategy: Classification of Sal actad Task 2/4 Lab Prograas/Projects by Rassarch Approach. 



Raeearch Approach 



Primary Data 



-Primary Soupc* of Oata 

Secondary Analysli 



Intaractlon Documentation 



A. Applied 
Rasaarch 



6. Evaluation 



C. Policy 



Planning 



RBS :CSIP/ Applied Ras/Pol1cy Studlas 
M1d Atl Council atudy of ID H Schs 

NW 4.5: Ras on Classroom Assessment 
thraa studlaa col loot primary data 

NW 4 •: Taa Turnovtr 1n Rasarvatlon 
Schools 

NE/I: Laadarshlp for Sch Impr* 
Study of Urban High School • 

FWL: Study of taa racrultaant ft 
ratantlon 1n CA 



NW 4.5 Ras on Classroom Asaassaant 

Ona study 1a synthasls of ras 
NW 4.13: Sch Improvement Res Synth 
NCREL: rev of ree on etrategles 

for Improving Instruction 
NCREL: 10 "Interpretations of ree 

or. teacher rolee" ft 4 Inter ps of 

case etudy research 
NCREL: 24 papere/aonograph on 

dimensions of thinking 
NCREL: Rev of Ht on Incentives 

for personnel 
AEL: Anal of longlt DB on family 

conditions ft student achievement 
AEL: Demog Study of Rural Smal 

Schools and Districts 
HcREL: Teacher Demand ft Supply 



SEIL: Model for tea research on 

writing (subcontract) 
NCREL: Invitational oomlnar on 

etrateglee for Improving 1nstr 

will produce mats for summer 1nst 
AEL: Classrooa Instruction Prog: 

Study group reports 



SEIL: Tracer etudy of SEED (subcon) 
RBSs Paper: Exemplary SO Practlcee 

Related to Info Collection ft Use 
McREL: Reeder Response: McREL Updete 
RBS: Studies of state mandated a1n 
competency testing 1n PA ft MD; 
Study of MD HS Orad Requirements 
FWL: Eval prof dev ctr 1n NV co 
: Eval career ledder syst 1n UT 
: Eval CSU Summer Bridge ft Inten 
slve Learning Experience Prog 
AEL: The Effectiveness of Study 
Groups ae an RftD Methodology 

SEIL: Taacher labor market etudlee 

SEIL: Longlt study of r mm curt 
RBS: Appl r . >s/Pol1cy Studies: Strat 

Planning Studies 1n LEAs ft schs ; 

Coord of Ed ft Soc Services 1n.PA 



NW 2.1: Qtr software analysis 
reports 

NW 2.4: Item ft Test Info Ctr: 
Consumer Guides ft Guide to Item 
Banks 



RBS: CSIP/Org Eff/Sec Sch lapr: 

Papers Sec Schs ft Centre! Office 
NW 2.5: Reg conf on Collegia! 

Staff Oev (no proceedings?) 
NW 2.12: Rsglonal Nets for Sch lap 



SEIL: Supply Side of Tea Lab Mkt 

RBS: 3 pusUlon papers for P* lag 



Reporte of policy forums 



W 4.16: Ef c ect1ve Voc Ed: Use 
streteglc planning to work w LEAs 
toward new directions 1n voc ed 
McREL: North Oakota State Plan 
for Technology 



NW 4.12: Profile of Pacific Schs 
NW 4.15: The Automated Workplace: 

synthesis of previous research 
NW 4.16: Eff Voc Ed: lit search 
NE/I: Leadership for Sch Impr: sec 

analysis of 2 sch 1apr studies 
NCREL: Pev of lit on SO; dev frame- 
work for planning SD progs 
McREL: various strat plan projs 



NW 2.7: Rural Networking : supervlss 
ft monitor plan ft dev of collab 
arrangements between nets ft SIO* 

NW 2.17: Stete plans for working 
with and through SIOs 
W 4.4: Technology Forecast Panel 
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The reliability of coding lab programs and projects into the 
cells of this table may not be high, but the typology may still 
have heur 1st 1c value 1n illustrating a fairly wide variety of 
research approaches and methodologies used. Aga in . no claim is 
n^ae that this table represents a compl ete and comprehensive 
cl asslf icatlon of lab activities . 

While no attempt to review all of the work found in each cell 
1s being made, a few comments are 1n order. Examples were found 
for every cell of the table. Collection and analysis of primary 
data is still an approach used by labs. While there is no way of 
quantifying and comparing the level of effort for the three 
columns, it appears that most labs are more involved in secondary 
analysis rather than the analysis of primary data. In addition, in 
a number of cases Involving primary analysis, the work 1s being 
done by a subcontractor rather than by lab personnel. Interaction 
documentation constitutes an important form of know 1 edge product ion 
for labs. 

The evaljation category may include some projects not actually 
funded by OERI? 1t is not always easy to tell whether the work 
being reported 1s inside or outside the OERI contract. 
Devel opment /Demons t rat ion Approach 

Development and demonstration are also knowledge-based 
Improvement strategies, but they cannot be subcl assif ied in the 
same way as research approaches. The categories used in the stub 
of Table 14 are of a mixed bag. Development is divided into two 
types, that done by the lab alone and that done jointly with some 
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other organization. Promising practices is a knowledge-based 
approach, but it is based, at least in the first instance, on craft 
knowledge, not research knowledge. We will investigate this 
strategy in more detail below. Finally, Demonstration involves 
implementation and operation of arv innovation on site, usually but 
not always a local school, to document and illustrate the 
feasibility and effectiveness of the Innovation. Demonstration is 
a permissible exception to the "with and through" linkage strategy 
when SDOs are Involved that permits the lab to retain "hands on" 
experience . 

The column heads of the table differentiate types of materials 
as student materials , teacher material s . trainer mater ial s , and 
administrator materials . This is probably the least reliable 
classification of al 1 because it is frequently difficult to tell 
for whom the materials are intended. Projects involving "program 
descriptions" are generally classified as "administrator 
materials," although the programs might well be oriented toward 
teachers or trainers. 

Again, no claim is made that this table is comprehensive; in 
addition, generalizations are hazardous. It would appear, however, 
that the focus is on student mater ial s 1 ess often than on other 
types of materials. This represents a sharp contrast to the former 
focus of older labs when the primary improvement strategy was 
curriculum devel opment . 

When development is Involved, it appears that labs are more 
likely to proceed on their own than to engage in joint activity. 
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Teble 14. Improvement Strategy: Claaalf 1ctt 1on of Select ed Leb Task 2/4 Prog raea/Pr ejects by DtvtlopMnt/DMonstntlon Approach. 



Typo of "OovoTopoont 
Demonetretlon 



Student MeteMsls 



Typo of Materiel a --------------------- 

Teacher Hotorlsls Trslnsr Hotorlsls Ade1n1atrator Hstorlolo 



Loborotory 
Development 



NM 2.2t Too Ctr for Demo ft Trng: 
Collection of herd were ft software 



RBS: Special Populations Sen 

Ieproveeent Model 
NW 2. St Trng Ctr for Cleeerooo 

As sees stent (slso T of T ft pro) 
McREL: Hath scope ft sequence for 

rurel high schools 
McftELs Tootles Teacher Menus 1 
MCREL: Options handbook on 

etreteglee for 1netr lepr 



SEIL: Guide to Pvt Sector Reeource 

for Principal Effectiveness 
SEIL: Effective Meth K-5s workshop 

©del, handbook, video tope 
RSSt Organizing for Sec Sen Oev 
NM 4.7: Rea Baaed Sch Impr 

4. let Eff Voc Edttrng eetorlele 
NCRELt Prof Devi Manual for 

ateff dev plsnnlng 
NCRELt Models/ Indies tors of 1nstr 

loedorehlp 
NCREL: Res eynth on toe od 
FVLt Outcooo Baaed Ed: Casebook. 

Hods Is of Inetr Org 
AELt Claaare Inetr Prog: Devi 

Turnkey Training Activities 
McREL: How to Teach Teachers 



SEIL: 2 duet ere of eodulee: 

Competencies of High Perf Pro 
RBSi Spec Pope Sch lepr Model 
NM 2.7: Rurel Networking: Trng 

eete for SlOa re role 1n 

developing rurel note 
NM 2.B: Trng Ctr CR Assess: trng 

eetorlele for prlndpele 
NM 4.B: Ado Guide for Too 

Turnover 1n Reeervetlon Sche 
NM 4.13: Sch lepr Res Synth: Oev 

electronic coaaun1cst1on ayetee 
RBS: Institutions! Anelyele: 

01 rectory of Instrueenta 
McRELt Ed loedorehlp trng eodulee 
McREL: Planning guide for 1epl 



B . Joint Development 



NE/Ij Modulee for Thinking Skills 
Progrse (Jt w Herverd P> *»f ) 



NM 4. IS: The Automated Workplace: 
dev corporate mentoring ft field 
plscement program (w bus col lobe) 

FML 3.1: Eff Suppt Bog Teas, w/'-A 
dev ft teet trslnlng prog 

FML: Devi ft teat Teaching for 
Scientific Literacy 

McREL: ASCO tope on etreteglc 
planning 



NM 2.B: Colleglel Tea Trng: Mentor 
Teacher H sod book (with SlOa) 

FML 3.1: w/Loa Angeles SO. dev 1 
eete of reeource mete for mentor 
teaching 



ftSS: LORE (Loedorehlp for Org 
Renewal ft Effect) w/Ph11s 

RBS: Sch 0 Eff Audit (w NJ Sch Bds 
Assoc) 

NM 2. 7 i Rural Networking: Oev rur 

networking models 
NM 4.12 Pacific R&D Oe ebeee 
AEL: Reg L1o1oon Ctrt lodol of 

home-echoo-com pertnerehlpe 

w/Norfolk community 



C. Prom1a1ng Prsctlcee 



SEIL: SEED (Software Evel Exchange 

aee Trscsr Study 
NM 2.1: RICE deteboee on couraewari 

evaluatlona (Jt w SEAs) 
NM 4.14: Prom Pracr for At Risk V 
McREL: Pre1r1e Minds 



NW 2.4: Item ft Teet Info Center 
NW 2.10 Eff Prsctlce m Indlsn Ed 
NM 2.12: Reg Note for Sch Iapr 
NM 4.1: Computer ft Curr Studios: 

caeee of Integrating eoftwero ft 

curriculum 
NM 4.12: Psdflc Eff Pr set Ices 
NM 4.13: Sch Impr Roe Synth: 

Exemplsry Pr set Ices 
NE/I: Identified print, prog ft 

hun «• r o* our css on 6 topics, * 

resource b1bllogrsph1es 
NCREL: Hen d book of options for 

evslustlng computer sppHcetlons 
NCREL: Collect ft collets info on 

cutting edge eppHcetlone of tech 

to students st risk ft HOTS 
NCREL: Collect, etudy ft snelyxo 

SEA ft LEA prsctlcee, policies. 

proge on personnel incentives 
FML 2.1: Outcome Baaed Ed producte 

school prof Ilea ft rseourcae dlr 



SEOL: Theme B: Applying Technology 

to Sch Impr ??? 
NM 2.B: Colleglel Tea Trng: Search 

for PPa for Induction ft long 

term prof Jev (w SIOs) 
NM 4. IB: Eff Voc Ed: study 

eff prsctlcee st ech sites 
NE/I: Tee Dev: Guidebook 
NCREL: Prof Oev: project in 2 

etetee to 10 distinctive SO proge 
FWL 3 I. Eff Sup»it Beg Tee. w/ CSU 

dev eeeebook. Close to the Cleera 
AEL: Prof Prop ft Res: VA ft MV. 

directory of tee trng resources 



0. Demonetretlon 



MM S.St Toe Ctr for Demo ft Trng: 

Domes of herd ward ft software et 

headquarters ft in field 
NM 4.14. PP for H1 Risk V: Help 

selected LEAe plsn sit leern envli 
McREL: Technology Demonetretlon 

SO DAT 



NE/I t Agreements with t SEAs ft 

11 LEAs for demo echo. At Risk Y 
FML 1.2: Self-Dlr Leern 1n Sc1 
AEL: School Excellence Sites (SES) 
McREL: ACCESS: Preparation for 
SAT and ACT 



SEDL: lepr Sch ft CR Productivity: 
outcomes for trng SID fedHtetor 
5 sites w SID partners 

NM 2.13: Psdflc Locel Csp STdg: 
Trsln trslners ("locel cedree") 

FML 3.1: Eff Suppt Beg Tess, w/CSU 
Support ft document pilot demoe 1n 
clinical supervision of bog toee 

McREL: Tactics reining for MO 
Loedorehlp Academy 



SEDL : Thome A: Pertnerehlpe with j 

Perento. Business, ft Communities i 
SEOL: Theme C. Tee/Adm Perfmence 

Aaaeeammnt Syeteme 
RBS: Compendium ech 1mpr indlcetor 
NM 2.B: Sch 1mpr thru Eff Tee 

Evel: Five Keye to Growth 
NM 2. IS: Need 10: trng mate on 

problem analysis 

M 2.14: Pvt Sector Networking: 

Linkage models 
NE/I t Tee Ed Reform Datebeee 
NE/I: Loedorehlp for Sch Impr: 

Directory of Resources 
FML 1.1: Profile of etete 

eaaeaament programs 
FML S.1 Eff Support Beg Tees w/CSU 

Policy ft Prog Casebook 
AEL: Sov/Ade: etudy group producte 



RBS: Sec Sch 

working w tei 
FML l.fi: Prom 
for ech 1mpr 



Impr: demo eltee, 
aee fr cent off 
>te Diet Opt lone 
w/ Clerk Co NV 
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On the other hand, the promising practices strategy is almost 
always an approach requiring work with other organizations. 

Approaches to Staff Development 

i he 1 ast two sections have provided an overview of lab 1 inkage 
and improvement strategies and subtypes of interorganizational 
collectivities, and rjsearcn and development/demonstration 
approaches. We would now like to get a more holistic view of 
contrasting strategies . 

While all laboratories identified staff development as an 
important educational focus, there was considerable variation in 
definition of the problem and improvement and linkage strategies 
employed. Because they divide naturally into three subgroups, 
profiles of selected of staff development programs appear in three 
separate tables: 15, T6 , and 17. Table 15 shows three labs that 
sel ected a trainer of trainers approach mi xed with an important 
knowledge production component. Table 16 shows three labs that 
al so had a knowledge production component but combined it with 
information dissemination rather than training. Finally, Table 17 
shows the other three labs that adopted three different strategies: 
a nearly pure knowledge production approach ( FWL ) , a networking and 
information dissemination approach (McREL), and a comprehensive 
approach (NWREL). Note that these are selected programs, chosen to 
reflect variations in approach. No claim is made that the program 
sel ected for any lab necessarily ref 1 ects its t otal approach to 
staff devel opment . 
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Table 15. Selected Approaches to Staff Development : Knowledge Production end Training Trainer* 



Design Feeture 



Program/ 

Project 

Heme 



AEL 



Classroom Instruction Prog 
Training opportunities 
b. TURNKEY TRAINING* 



Cooperative School Improvement 
I nst r uct 1 one 1 I mpr ovement 
TURNKEY TRAINING 



Problem 
Definition 
(Educations 
Focus) 



6 . Improve- 
ment 

Stretegy 



Linkage 
Stretegy 

1. Collabor- 
ators end 
clients 

2. Services 



3. Provision 
for 2nd 
tier 

D. Impect 
1st level 
and level 
3rd level 



E . Remarks 



174 



PROBLEM DEFINITION 

1. Need: to Improve teaching 
* learning 1n region 

2. 3 principles of action: 
o Work with end thru 

o Pr1v1de R*D-based Info 
o Provide opportunities for 
teas to Increase kn 



A. PROBLEM DEFINITION 



B. IMPROVEMENT STRATEGY 
o Using methodology of RDIS 
synthesize reseerch on st 
dev, math * sc1 , end oral 
* written comm a reeding 
o Produce 4 training 
pkgs 1n 36 end 9 1n 87 



C. LINKAGE STRATEGY 

1 . Collaborators/clients 

o NEA stete affiliates ere 
sponsors; trainers * SDO 
personnel "charged with 
staff development" 

2. Services 

o Training workshops 



3. Second tier 
o Ad hoc 



D. IMPACT 
o 120 1n 6 workshops 1n 8Y 
o 2)1 by II of above 



E. REMARKS: Part of larger 
program. *Also provides 
turnkey training to NEA * s 
Mastery 1n Learning project. 



R8S 



EFFECTIVE K-5 MATHEMATICS 
REMEDIATION 



|A. PROBLEM DEFINITION 

Need for more teecheri 
with four competencies 
Goals : 

Communicate theories 
end strategies of 

Instruction 
Train trainers of 
trainers 



B. IMPROVEMENT STRATEGY 
o Produce/refine trelnlng 
peckeges 1n 3 content erees 
thinking skills, elementary 
excel lence, end Instruction! 
supervision . 



C. LINKAGE STRATEGY 

1 . Col leboretors/cl 1ents 
o Collaborators: SIOs 

o Clients: staffs of LEAs , 
SIOs, prof assocs, & SEAs 

2. Services 

o Turnkey training 



3. Second tier 
o Ad hoc 



0. IMPACT 
o "...the use of materleli 
wss more piecemeal then 
programmatic' 1 



E. REMARKS 



SEIL 



IMPROVEMENT STRATEGY 
Knowledge bese 1s 
reseerch synthesis on four 
competencies teachers 
should heve. 
Dev products: workshop 
model, hendbook ft video ta 



. LINKAGE STRATEGY 

. Collaborators/clients 

o SEAs sponsor end nominate 

participants 
o SDO personnel ere trainees 

2. Services 

o Pr.^lop non-content parts 

of workshop model 
o Treln trelners 

3. Second tier 
o Ad hoc 



0. IMPACT 
o 20 1n 1 workshop 1n '87 
o 32 by 2 of above 



E. REMARKS 

Was subject of 3rd party 
tracer study. 
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KP/TT Approaches to S taff DoyeTogmgnt . 

AEL, RBS, and SEIL all designed trainer of trainer programs 
utilizing materials developed for that purpose. The problem 
definitions of AEL and SEIL combine pragmatic and theoretic 
rationales. SEIL had the narrowest focus:, namely K-5 mathematics 
remediation. AEL and RBS included both curricular foci (math and 
science, oral and written communication, thinking skills) and more 
general staff competencies (research on staff development, 
elementary excellence, and instructional supervision). For an 
improvement strategy, both AEL and SEIL produced research syntheses 
as the knowledge base for their training materials. It is not 
clear whether RBS usad a research or promising practices approach; 
it specifies "produce/refine training packages in three content 
areas." Both AEL and RBS refer simply to "training packages." SEIL 
identifies three development products: a workshop model, a 
handbook, and a video tape. 

The labs also varied in their linkage strategies. They tended 
to perceive staff development as an internal SDO function rather 
than one for which school districts look for outside assistance. 
Although state NEA affiliates (AEL) or SEAs (SEIL) acted as 
sponsors , the c 1 ients were SDO personnel charged wi th staff 
development responsibilities. RBS trained both SDO and SI r 
personnel . 

Cu, iously, it appears that none made specific provisions for 
the training that clients were expected to provide. Apparently 
they believed that this would be taken care of by selecting 
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individuals with staff development responsibilities (AEL) or from 
SIOs with improvement roles (RBS) . The third lab (SEIL) f relying 
on nominations from SEAs, found that some of Its trainees were 
principals with no training responsibilities or teachers who 
perceived themselves as weak in the subject matter and therefore 
unqualified to train others. Perhaps it is difficult to influence 
established and active inservice programs, and labs may not have 
much leverage concerning second and third tier training, but it 
seems that labs need to give further thought to these next steps 
before they employ a TT strategy. Impact data indicate 
considerable attrition from one tier to the next. In the case of 
AEL, although 120 trainers were trained in six workshops in 1987, 
only 19 of these provided training to others. These 19 trained 236 
in the second tier. SEIL held only one workshop for 20 trainers in 
1987. Only two of these trained 32 at the next level. Clearly 
reduction 1n the attrition rate could greatly Increase the leverage 
achieved with a TT approach. 

KP/Inf ormat 1on D1ssem1nat 1on Approaches to Staff Devel opment 

A second group of three labs (NCREL, Nc/I, and SEDL) combined 
knowledge production with information dissemination (see Table 16). 
Problem def in1t 1ons tended to be pragmatic . All three 1 abs 
empl oyed a promising practices improvement strategy, al though NCREL 
included a component for designing and testing training materials, 
and NE/I indicated an interest in using adul t 1 earning theory . 

No collaborators wre identified for the linkage strategy. 
Clients of the information dissemination efforts were more general 
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Table 16. 



Design Feature 



f!!! 0 !!^^!!!^*." St * f ^ 0#v " 1o P"» nt! Knowledge Pr -.^jon and Information D1 a aaml nation 



Program/ 
Pro J act 
Nama 



Problem 

Daf 1n1t1on 

(Educational 

Focus) 



NCR EL 



Improvamant 
Stratagy 



Program for Professional Dav 

8 1 STAFF DEVELOPMENT 



A. PROBLEM DEFINITION 
o Roform reports call for 
Improved insarvica for 
both beginning and 
axparisncad taachars 
o Noad now goals and modal* 
& Innovative delivery & 
Implementation strateQlas 
o Now staff dav legislation 



|A. PROBLEM DEFINITION 
o The variety of groups that 
provide staff development, 
need support 



C. Linkage 
Strategy 
< Collabor- 
ators and 
clients 

2. Services 



. Provision 
for 2nd 
tier 

Impact 
1st level 
2nd level 
3rd level 



E . Remarks 



B. IMPROVEMENT STRATEGY 
Study and catalog staff 
dav programs & practices 
Design end test trailing 
materials 

Profile practices 1n 
schools of education 
Prapara papers 



C. LINKAGE STRATEGY 

1 . Col laboretors/cl 1ants 

o Clients « "Dissemination 
partners" (m1sc KPOs a 
SIOs) ^Inferred] 

2. Services 

o Information dissemination 
[Inferred] 

3. Second tier: 
o Ad hoc 



D. IMPACT 



E. REMARKS 

Sourca: "Addendum to Full 
Service pien FV 1967" ; 
Project could not be 
Identified 1n 4th Qtr Report 



NF/I 



Priority Area 2: Teacher 
Development 

2.1 PROFESSIONAL DEV MODELS 



A. 3 PROVIitE TA [to SlOa] 
TO HELP THEM PREPARE ELEM 
TEACHERS & ADMS FOR PARENT 
INVOLVEMENT 



B. IMPROVEMENT STRATEGY 
o Apply adult 1 ear , 1m 
theory 

o Develop effective practices 

f 1 1 e/d 1 rectc ry/gu 1 debook 



LINKAGE STRATEGY 
1 . Collaborators/clients 
o Clients « LEAa , ISAa 
and iHEa 



2. Services 
o Information dissemination 



Sacond tier: 
o Ad hoc 



SEDL 



A. PROBLEM DEFINITION 
Partnerships with parenta can: 

o help with problem of ahrlnklng 
resources fc rising expectations 

o Improve climates for learning 



8. IMPROVEMENT STRATEGY 
o Compile bibliography end 
directory of materials 
on parent Involvement 



IMPACT 



E. REMARKS 

Original mora comprehensive 
program sea leu down 1n second 
ywer . 



C. LINKAGE STRATEGY 
1 . Col laboratora/cl 1ents 
o Clients are trainers 1n 

state-level tea ad assocs, 

SEAs, IHEs , etc. 

. Services 

o Develop procedures and 
recommendations for 
providing tech o^litanci 

• Sacond tier 

o Ad hor 



IMPACT 

Six state-level workshops 
for 165 representatives. 
No data on second and 
third tiara 

E. REMARKS 

3ne piece of programmatic 
effort on parent Involvement. 

Described aa TA rather 
than training. 
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than targeted. NCREL apparently sent its information to all its 
"dissemination partners/' which included a wide variety of KPOs , 
SIOs and SDOs. Similarly, NE/I sent information to LEAs , ISAs, and 
IHEs. SEDL's effort was more targeted, aimed at trainers in 
state-level teacher education associations, SEAs , and IHEs. The 
exact nature of SEDL's work is hard to categorize. It describes it 
as technical assistance (rather than training or information 
dissemination). Six st^te-level workshops were held for 165 
"representatives" who were given information about the SEDL 
moterials on parent Involvement and "orocedure^ and recommendations 
for providing technical assistance." It seems closer to being a 
trainer of trainers program. 

No data on impact are provided, either in terms of adoption or 
use of promising practic3s (NCREL and NE/I) or provision of 
technical assistance by those receiving the SEDL materials. 
Mlscel laneous Approaches to Staff Development 

Finally, three labs had unique approaches to staff development 
(FWL, McREL, and NWREL). Profiles of these programs are found in 
Table 17. 

FWL. The approach used by FWL appears to be almost exclusively 
a knowledge production effort. The Los Angeles School District is 
a very large LEA that hires -over* 3,000 new teachers each year. The 
FWL 1s focused on development of a mentor program whereby 
experlencea teachers provide support for th«5se beginning teachers. 
This development effort was a collaborative one with the 
Professional Devel opment Center and produc >d three resources : a 
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Table 17. Sol acted Approaches to Staff Devopmont: Miscellaneous 



Design 
Feature 



FWL 

(Knowledge Production) 



McREL 

(Networking a Information D1si 



NWREL 
(Compr ehons 1 ve ) 



Program/ 

Project 

Name 



A. Problem 
Definition 
(Educational 
Focus) 



Prof Prep a Oevelopmert 

EFFECTIVE SUPPORT FOR 
BEGINNING TEACHERS 



Subtask 2.3s Network a Broker 

People and Resources 
2.3.2: Foster Interpersonal 

Networks: STAFF DEV NETWORK 



Improvement 
Strategy 



C. Linkage 
Strategy 

1. Collabor- 
ators and 
clients 

2. Services 



3. Provision 
for 2nd 
tier 

). Impact 
1st level 
2nd level 
3rd level 



E. Remarks 



A. PROBLEM DEFINITION 

1. Need: Lcs Angeles hires 
over 3,000 new teachers 
per year 

2. Goal: Assist Los Angeles 
Prof Dev Center train 
900 mentors 



A. PROBLEM DEFINITION 

1. Large region with goographl 
diversity, sparce populatlo 
and Isolated professionals 

2. Limited Infrastructure and 
declining local and state 
resources 



B. IMPROVEMENT STRATEGY 
Jt dev w/ LEA of 3 resources: 
o Leader's Guide for 

Preparing Mentor Teas 
o Mentor Teacher Casebook 
o Intern Teacher Casebook 



6. IMPROVEMENT STRATEGY 
o Ad hoc: Identified through 
network Interaction 



C. LINKAGE STRATEGY 
1 . Collaborators/clients 
o Professional Development 
Center of Los Angeles LEA 



2. 
o 



3. 
o 



Services 

Development + publication 
and sale of resource books 
beyond Los Angeles 
Second tier 

Trng mentors 1s regular 
responsibility of partner 



C. LINKAGE STRATEGY 

1 . Col laborators/cl 1ents 

o Individuals w staff dev as 
major rasp, fr SIOs, SDOs , 
and private consultants 

2. Services 

Network creation & sponsorship 
o Semi-annual meetings 
o Coordination mechanisms 
o Steering committee 

3 Second tier: 
o Ad hoc 



D. IMPACT 
o Not specifically stated; 
Presumably 3,000 mentorn 
trained by PD ctr staff. 



E. REMARKS 



D. IMPACT 

o Increased SEA communication 
o McREL participates 1n SEA 

1nserv1ce programs 
o SEA exchange of personnel 

E. REf ARKS 



2.5 COLLEGIAL TEACHER 
TRAINING 



A. PROBLEM DEFINITION 

1. Too few excellence reforms 
focus on Instruction level 

2. Goals: 

o Establish prof dev network 
o Build capacity of SIOs to 

train area col laboratlves 
o Diss findings fr kn bases 

a col lab efforts reg a net 

B. IMPROVEMENT STRATEGY 

o Synthesize res on effective 

schooling* tea, a prof dev 
o Dev handbooks for mentoring 

thru collab design confs 
o Find promising practices 

for Induction and long-term 

prof development 



C. LINKAGE STRATEGY 
1. Col 1 abonrort/c 1 1 tntt 
o Collaborators: SEAs a IHEs 
o Clients: consortia of SDOs 



2. Services 
o Info: diss hbooks; reg conf 
o TA to consortia 
o Jt trng with SIOs at demo 

sites a other SIOs 
o Demo sites 1n 3 States 
o Estab prof dev network 

D. IMPACT 
o No data 



E. REMARKS 
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Lecider's Guide for Preparing Mentor Teachers, a Mentor Teacher 
Casebook, and an Intern Teacher Casebook. The linkage stratagy was 
the use of these materials by the Los Angeles School District in 
Its regular staff development program. Presumably over 3,000 
mentor teachers were trained by the Professional Devel opment 
Center. In addition, the materials were made available for 
purchase by other districts. In this case joint development with a 
large SDO virtually assured implementation. 

McREL . McREL's Staff Development Network Program is probably 
the most opportunistic or the nine. (Other staff development 
activities are carried out in other programs.) It consists of 
creating and supporting a network of persons responsible for staff 
development in SEAs, other SIOs, SDOs , and private consultants. 
McREL*s support consisted of (1) organizing semi-annual meetings, 
(2) coordination mechanisms, (3) a steering committee, and (4) 
distributing related information. The resulting staff development 
Improvemert efforts were ad hoc, emerging from the interaction of 
network members. McPtL cites as impact increased communication 
among staff development personnel 1n different SEAs, McREL 
participation 1n SEA inservice programs, and exchanr ? of staff 
devel opment personnel among SEAs . 

NWREL . The staff development program of NWREL, Collegial 
Teacher Training, is the r,>ost comprehensive. It notes that toe few 
excellence reforms focus on the instructional level. The goals of 
the program are to establish a professional development network, 
build the capacity of SIOs to train LEA co" laborat i ves , and to 
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disseminate regionally and nationaly findings from knowledge bases 
and collaborative efforts. Three improvement strategies are used: 
research synthesis , handbook devel opmer.t , and selection of 
promising pract ices for both induct ion and long term professional 
development. For linkage, both SEAs and IHEs are listed as 
collaborators, although the nature of the collaboration is not 
specified. Interestingly, the clients are not individual LEAs but 
consortia of LEAs interested in improving staff development 
efforts. Given the small si7e of most Northwest districts, joint 
staff development efforts are apparently felt to be ,7,ore 
cost-effective. Specific services run the gamut, including 
regional conferences, dissemination of handbooks, establishment of 
demonstration sites 1n three states, joint training with SIOs at 
these sites and at other SIOs, provision of technical assistance to 
consortia of SDOs, and establishment of a professional development 
network. Unfortunately, no impact data are provided. 



Staff development- was recognized as an area of need by all 
laboratories. Some focused on the problems of inducting beginning 
teachers while others focues on Inservice for experienced teachers, 
or both. One or more labs used virtually all Improvement and 
linkage strategies. This would be a good topical area for labs to 
get together and compare notes on what works best and possibly 
design some comparative studies that would produce more systematic 
evidence . 



Comments 
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The Promising Practices Improvement Strategy 
The RFP enjoined labs to use R&D-based knowledge as the basis 
for its improvement efforts. When the labs employ a research 
synthesis strategy or develop materials designed with reference to 
some research knowledge base, 1t is clear that they are using this 
approach. When they search for and select promising practices for 
cataloging and disseminating, it is difficult to say whether the 
practices are R&D-based. 

hor some years, programs supported by OERI have attempted to 
gi^e equal weight to craft knowledge and R&O-based knowledge. 
During this same period philosophers of science have argued the 
relative merits of various paradigms such as positivism, 
behaviorism, action research, interactive research, and 
naturalistic inquiry. " These Issues are beyond the scope of chis 
study, but we would like to get a clearer picture of just what is 
Involved when labs employ a promising practices methodology. What 
kinds of search strategies are employed, and what criceria of 
selection are used? In what sense, if any, are the practices 
R&D-based? 

Most of the 1 abs em pi oyed a prom is ing practices approach in at 
least one project. Unfortunately, the information available for 
this study is not detailed enough to permit the kind of analysis 
employed for other Issues. For example, an action step for a 
project might * pecif y "Identify criteria for selection of promising 
practices," but the criteria used are not reported in the kinds of 
documents used for this study. Therefore, this section will be 



CHAPTER V TASKS 2 & 4 



Page V-4 1 



limited to identifying some of the programs and projects that 

employ this approach, examining some variations, and dijcussing 

some of the issues involved. 

The following is a list of lab programs and projects that 

a ppear to empl oy a prom is ing pr act ices approach : 

FWL. 3.0 Enhancing Teacher Qua! itv and Support for 
School-Based Improvement : 

3.3.2: Inventory preservice and inservice interests and 

act ivi t ies in the western region . 
3.5.1: Help districts assess existing school-level 
practices and capabilities. 

NCREL. B.l. Staff Development : Identify distinctive staff 

development programs in two states; describe data 

collection procedures and standards employed. 
Publ i cat ion in 1 988 . 

NE/I ♦ 3*1* Resources for Effective Classroom Practice . Six 
topical areas; publish resource bibliographies, overview 
articles, list of materials, and 11st of program 
descriptions . 

NWREL. Activity 2.4; Item and Test Information Center : Gather 
and com p 11 e tests , test ing ideas , assessment approaches , 
issues and Items; 

Activity 2.5; Collegia! Teacher Training : Conduct search 
for promising extant models for induction and long-term 
professional development . 

Activity 2.14; Private Sector Networking : Survey regional 
and national linkage models and classify by purpose. 
Activity 3 .4 Educational Standards and The1 r Impact on 
Organizations : Establish a repository of state and 
district standards developed throughout the Pacific and 
Northwest Regions, as well as selected standard-setting 
"models" Identified from across the nation (Task 3). 
Activity 4.7; Study of the Core Curriculum : Gather data 
and prepare prof 1 les of different models of "core 
curriculum" 1n the region. 

A ctivity 4 .13; School Improvement Research Synthes es : 
Identify, describe and make visible and available for use 
among practitioners information about exemplary uses of 
effective schooling practices for school improvements at 
the state, Intermediate district and local school levels. 
Activity 4.14; Promising Programs for High Risk Youth : 
conduct a regional search of alternative education 
programs that are effective at the middle school and high 
school le>/el . 
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Activity 4,18; Study of Effective Vocational Education : 
Identify through a literature review and contacts with 
leading R&D institutions applications of effective 
school ing research to vocational education . 

SEDL. A. Devel oping Educational Partnerships with Parents , 
Businesses . and Communities t 

A . 1 . 1 . Identify and describe promising programs/ 

practices of school -business partnerships. 
A.2.1. Identify and describe examples of new and 
promising parent involvement programs and networks. 

B. Applying Technology to School Improvement : Identify 
and descr 1 be Innovat i ve and promising c 1 ass room prac t ices 
and school programs that use technology. 

C. Improving Teacher and Administrator Performance : 
Identify and synthesize information on teacher and 
admlnls rator performance systems. 

SEIL. Project SEED (Software Evaluation Exchange Development 
and Disseminat ion ) : Indexed annotations on software 
evaluations made by pract 1t 1on 3rs trained in procedure 
and mailed to schools. 

A number of obser?.va t ions can be made about this array of 
promising practices programs and projects: 

o This methodology is applied not only to classroom 

practices but also administrative practices, testing, 

software, [academic] standards, and professional 
devel opment programs . 

o Some projects 1 ook for pract ices 1 inked to pr inc i pi es 
1n the research literature (e.g., NWREL 4.13 and 4. 
16) 

o One part of the FWL project seeks to enable SDO 
personnel to make better choices of practices rather 
than to provide the promising pract ices . 

What makes a practice R&D-based? It can acquire this status 
either deductively or Inductively. In the linear model, a practice 
1s developed on the basis of principles uncovered through research. 
Much development has been Inductive, however. Practices are 
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designed on the basis of professional experience or hunch and 
progressively refined through evaluation and redesign. Practices 
that are originally designed deductively may go through a similar 
process of testing and refinement. Whether a practice deserves to 
he called "R&D-based" depends on the degree to which it is 
logically consistent with research principles and/or subjected to a 
rigorous process of evaluation and redesign. 

Research Itself can be either deductive or inductive. The 
effective schools research has been inductive (for the- most part). 
Schools judged to be more effective on given criterion measures 
were found to exhibit certain' characteristics more than did schools 
judged to be less effective. But there was no evidence that these 
characteristics caused the schools to be more effective. When 
schools are changed to exhibit the characteristics of effective 
schools, this may constitute a deductive test of principles arrived 
at inductively. It may be that some of the labs' promising 
practices projects are of this nature. 

An Important evaluation question for the labs is whether their 
promising practices activities meet the test of logical consistency 
with research principles and/or the test of rigorous evaluation. 
Practices that are selected solely on the basis of practitioner 
testimonials belong in a different category of "craft-based " 
practices . 



Conclusion 

This chapter has analyzed Tasks 2 and 4, which represent from 
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60 to 78 percent of individual labs T effort. Although Task 2 

corresponds to the linkage strategy and Task 4 corresponds to the 

improvement strategy, we again found that all projects have three 

strategic dimensi ons : a 1 inkage strategy, an improvement strategy , 

and an educational focus. Projects - tend to be identified in terms 

of the strategic dimension that was the point of departure in the 

planning process . 

Further refinements were made in the conceptual framework for 

all three strategies. Starting with a set of programs dealing with 

improving administrative leadership, the educational or substantive 

focus dimension was further refined by observing varieties of ways 

the problem was defined. Some saw the problem in terms of the need 

for administrators to understand new bodies of research findings, 
others in terms of the isolation of administrators, etc. Labs 

used both theoretic and pragmatic rationales and focuseu on both 

curricular knowledge and teaching competencies. 

Linkage strategies had previously been analyzed in terms of 

knowledge base, interorganizatlonal arrangements (national and 

regior al collaborators), and interpretations of the "with and 

through" strategy. Turning to a set of Task 2 programs, the 

linkage strategy was elaborated by Identifying four types of 

inter organ izat ional col lectl vi ties (partnerships , col laboratives , 

consortia, and networks) and differentiating those of which ths lab 

was a member from those that we^e served by the lab as clients. 

Some tentative findings were: the collaborative (multi-purpose, 

b1-lateral) was the 1 east common form of inter org an izat ional 
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collectivity, and far more single-purpose arrangements 
(partnerships and consortia) than mult i -pur pose (collaboratives and 
networks ) existed . 

Improvement strategies previously had been described in terms 
of the know 1 edge base em pi oyed (research , products , and pract i ces ) , 
information services provided, and the KP activities involved 
(research, evaluation, policy studies, or development). In this 
chapter a set of Task 4 programs was further studied by subdividing 
research into applied research, evaluation, policy studies and 
planning? and coding the primary source of data (primary data, 
secondary analysis, or interaction documentation). Examples of all 
approaches were found. Under the heading of "development/ 
demonstration approach," four categories were recognized: lab 
development, joint development, promising practices, and 
demonstration. Programs in each were classified by types of 
materials (student, teacher, trainer, and administrator materials). 
There was much less activity involving student materials than the 
other kinds. 

All labs had programs or projects dealing with staff 
devel opmen t . Three labs used a trainer of trainers approach 
combined with the production of training packages. Labs tend to 
see staff development as internal LEA function and identify the 
final trainers 1n the chain to be LEA personnel, especially 1n 
large districts. However, they of'uen work with SIO personnel in 
early stages of the chain. Trainer of trainers programs were found 
to be weak in making specific provisions for first wave trainers to 
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follow through and train others. 

Three other labs combined knowledge production with an 
Information dissemination strategy in seeking to improve staff 
development. Evidence for the impact of these approaches was 
sparse . 

The other three labs had different approaches to staff 
development. FWL was largely a joint development effort with the 
Los Angeles public schools. McREL combined networking with 
Information dissemination, while NWREL used a comprehensive 
strategy . 

It was suggested that staff development was an area in which 
useful comparative studies or "strategic research" might be 
conducted . 

Finally, the "promising practices" approach to improvement, 
which involves collecting information about extant programs, was 
singled out for special analysis. It was noted that it may or may 
not be an P&D-^ased strategy, depending on whether the practice 1s 
logically consistent with research findings and/or subjected to a 
rigorous process of evaluation and redesign. The promising 
practices methodology was applied not only to classroom practices 
but also to administrative practices, testing, software, standards, 
and professional development programs. 
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CHAPTER VI 
SUMMARY AND CONCULUSIONS 

Nature of the Study 

This study has described the nine regional educational 
laboratories supported by the U. S. Department of Education's 
Office of Educational Research and Improvement. The labs are 
operating under five-year contracts for 1986-1990. The study has 
analyzed lab-produced documents describing lab operation for the 
first two years of this period, highlighting the status of the 
program at the end of the second contract year. These were largely 
management documents (plans, annual reports, sel f -assessment 
reports, etc.) rather than program documents. 

These aspects of the study, dictated by the request for 
proposal, impose certain limitations on the analysis. The reports 
vary greatly in detail and specificity. Consequently it is 
poss i ble in some instances that a 1 ab may be coded as not 
exhibiting a certain characteristic or undertaking a given kind of 
activity when that is not the case. Such information may exist in 
some document not examined, or if t he info r ma 1 1 op wa s p r o v i d ed i t 
could have been missed in the analysis. This possibility was 
considerably increased because several lab reports lack continuity. 
For example , a hierarchical program structure with every program 
project assigned letters <*nd/or numbers would be set out in a plan 
but would be abandoned or changed in later documents. Having this 
report reviewed by both the OERI institutional liaisons and the 
labs should hel p overcome such problems . 
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Some readers may doubt the credi bi 1 i ty of lab produced 
documents. If one discounts occasional passages of self- 
congratulation, the factual information can be accepted as 
reasonabl y val id . There are many exam pi es of candor in discussing 
problems and how they were dealt with. If OERI did not believe 
these reports contained useful information it would surely not have 
commissioned this study. 

Conceptual Frameworks 

One way that this report has attempted to advance our 
understanding of lab operations is by borrowing, adapting, or 
inventing conceptual frameworks for profiling and comparing lab 
programs and projects. Hopefully they will prove useful in 
designing further studies of lab strategies. 

The RFP governing the lab competition has, of course, been a 
major point of reference (NIE 1984). In that document the mission 
of regional educational laboratories was specified by the task 
structure and a set of "statements" (see Appendix A). All lab 
programmatic efforts were found to have three strategic dimensions 
that can be derived from the RFP : an educational focus , an 
improvement strategy, and a linkage strategy. The major difference 
is that, whereas the RFP specifies a linkage strategy in Task 2 and 
an improvement strategy in Task 4, this analysis has demonstrated 
that programs and projects 1 isted under these tasks exhibit all 
three strategies. This thrss-dimens Jonal schema has provided the 
major framework for the analysis of all lab programmatic 
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activities. Each dimension has, in turn, been further broken down 
into more detailed classifications of tactics, as follows: 

( 1 ) Every (non -management ) program and project has a 
substantive focus. What aspect of education is being addressed? 
The terminology used for this strategic dimension varies in 
different chapters, but the most generic and appropriate name is 
probably educational focus . In the context of needs assessment 
this dimension was examined as educational need or problem. In the 
context of state policy analysis it was examinpd as policy issues 
and domains . An adaptat ion of a c 1 ass if i cat ion of pol icy issues or 
domains used by FWL and derived from one by Mitchell and 
Encarnation (1983) was found to be reasonably comprehensive and 
mutually exclusive. In the context of Tasks 2 and 4 educational 
focus is examined 1n terms of variations 1n problem definition, 
including a distinction between theoretic and pragmatic rationales . 
Clearly, the specifics of how a problem is defined will have a lot 
to do with the selection of other strategies and tactics. 

A recapitulation of the categories used to descr 1 be the 
educational focus follows. These and other classifications in this 
chapter update and consolidate conceptual frameworks used in the 
analysis . No one analysis 1n the body of the report used the f ul 1 
set of categories . 

EDUCATIONAL FOCUS 

Problem definition 

Theoret 1c rat 1ona 1 e 
Pr a gmat 1c rati ona 1 e 
Educat 1ona 1 need/problem/pol icy/ is sue doma in 
Bui 1 dings and fad lit ies 
Curriculum and instruction 

ERJC ly, 
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Organ 1zat ion and governance 

Prog ram def i n i t i on 

School f inance 

School personrel 

Student type 

Student test ing 

(2) All programmatic activities nave an improvement strategy . 
Labs are limited to knowledge-based improvement strategies by the 
RFP (in contrast to strategies based on providing funds, changing 
regu 1 at ions , etc) . This dimens i on focuses on the types of 
knowledge used for educational improvement and how it is obtained 
and processed. Some facets of knowledge-based improvement 
strategies examined include knowledge bases used, information 
services provided, and types of KPU, including sub-types of 
research approach and primary sources of data. 

A recapitulation of the categories used to describe the 
improvement strategy follows: 

IMPROVEMENT STRATEGY 
Knowl edge base 

Research findings, by field (ad hoc) 
Ef f ect i ve school s 
etc . 
Products 
Extant 

To be developed 
Pract i ces 
Extant 

To be compiled 
Information services 
Database devel opment 
Reports based on databases 
Knowledge production to be undertaken 
Approach 

Research 

Eva 1 uat i on 

Pol icy studies 

PI ann ing 

Devel opment 
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Primary type of data 
Primary data 
Secondary analysis 
Interact! on documen tat i on 



(3) All programmatic activities have a 1 inkage strategy . Labs 
are sociological ly marginal organizations operating in a loosely 
coupled configuration of educational units. Since education lacks 
system characteristics, there is no assurance that new knowledge 
and practices will flow through it efficiently. Labs are in the 
position of having to cont inually negotiate their role in the 
organizational infrastructure. Consequently all programs and 
projects must specify a strategy and tactics for linking to other 
educational organizations. The RFP called for an indirect "with 
and through" strategy as the dominant mode of operation, i.e.,. labs 
woul d work pr imar i 1 y with and through other service improvement 
organizations (SIOs) rather than directly with service 'elivery 
organizations (SDOs). In this context, the conceptual framework 
was further elaborated to specify knowledge bases, 
Interorganizat ional arrangements, and communication modes. 

A recapitulation of the categories used to describe the linkage 
strategy follows: 

LINKAGE STRATEGY 

Knowledge base 

Research findings, by field (ad hoc) 
GDPA study (Louis et al 1984) 
DESSI study (Crandall and Loucks 1983) 
etc . 

Ord 1nary knowl edge 
Products 
Extant 

To be developed 
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Practices 
Extant 

To be developed 
Inter organ izati on al arrangements 
Col laborators 

National collaborators 
Other labs 
University centers 
Nat ion a 1 associations 
Other 

Regional col laborators 

KPOs 

S'Os 

SDOs 
CI ients 

Pr i mar y 

Chiefs 

Other SEA 

State government 

State-1 evel associations 

IHEs 

ISAs 

SDOs 

Cross sect ion 

Other 
Range 

Nar row 

Med ium 

Broad 
Type of client 

Organizat ions 

Ind 1 vi dua 1 s 

Networks 

Mixed 

Interpretation of "with and through" strategy 
SIOs arR clients 

SIOs are col laborators /sponsors , SDOs are cl ients 
SDOs are collaborators and/or clients 
Type of interorganizat ion collectivity (IC) 
Lab is member of I C 

Partnership 

Consortium 

Col laborat ive 

Network 
IC is client of lab 

Par tnershi p 

Co sortium 

Col laborat ive 

Network 
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Communication mode 
Approach 
Reg ional 
State-by-state 
Type 

Information services 

Search and ratrieval 

Publ ications 

El ectronic services 

Present at icn meet ings 

Br i ef ings 
Tec hnical assistance 

Telephone, mai 1 

In- person consultations 
Prof ess ional devel opment 

Tra ining workshops 

Tra i ning of trainers 
Other 

Part ici pat ion in inservice programs 

The conceptual frameworks presented above deal with lab program 
activities. Other frameworks apply to organization, governance, 
and planning activities, but only those for planning activities 
will be reviewed here." 

The planning process impl ic- t in the RFP requirements and 
exhibited by the labs was a version of strategic planning. Its 
major components are needs assessment, regional capacity 
assessment, self -assessment , and laboratory plans. 

Needs assessment might be structured in terms of tne 
classification of policy domains presented in the educc : :ional focus 
framework. Needs assessment has tended to be pragmatic in orien- 
tation. Consideration might given to structuring needs assessment 
along theoretical dimensions. For example, how are regional 
schools distributed in terms of charact prist ics of effective 
schools? Of course labs are dependent on the availability of good 
Indicator data for this kind of information. Several labs did 
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collect needs survey data using the Grunig communication theory, 
which seemed to be useful in determing the degree of concern with 
specific issues. A number of meanings of "need" were noted, 
and the concept should be clarified for further needs assessment 
efforts. Further, needs can be specified at different levels of 
generality. A level midway between the generic functions of 
education and the level of technical design was recommended. 

The weakness of regional capacity assessment was noted, 
together with the failure to use the methodology developed in the 
ARROE Study (Frankel et al 1979; Lehmlng 195?). That study 
collected information on educational organizations perform, ng 
research and research-related activities by type and including 
purpose, areas of primary educational R&D work, size of staff, 
specialities of professional staff, amount and source of funds with 
breakdowns b^ educational level and R&D function, and types of 
publications and dissemination activities. Adaptations of this 
framework to make it commensurate with other components of the 
strategic pi ann ing model wou Id be necessary . 

A review of the quest ions addressed in sel f -assessment 
activities revea ]r»d two types : criterion and purpose questions . 
Criterion questions could be grouped under the three strategic 
dimensions discussed above in addition to a fourth for organization 
and management . 



CRITERION EVALUATION QUESTIONS 
Educat i on a 1 focus 
Rel evance 
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Improvemant Strategy 

Qual i ty 

Uti 1 ity 

Impact/effect 

CI lent satisfaction 
Linkage strategy 

I nterorgani zat ional arrangements 

Reaching audience 

Lab vis i bi 1 i ty 

Indi rect strategy 
Organization and management 

Efficiency 

Capacity for sel f -assessment 

PURPOSE EVALUATION QUESTIONS 
Improve program 

Improve organ i zat ion /management 

Improve sel f -assessment 

Devel opment staff 

Ac hi eve contract compl iance 

Increase kn owl edge 

At a more general level, a variety of models or images were 
employed in sel f-eval uat ions . 

EVALUATION MODEL OR IMAGE 

Systems/CIPP 
Quality control 
St rat eg i c pi anning 
Goal free evaluation 
Eva 1 uat or as educator 
I nst i tu t ional renewa 1 



Lab plsns are expected to flow from these needs-sensing, 
capacity assessment , and sel 1 -assessment activities . One woul d 
therefore expect the plans to use the conceptual frameworks related 
to each of these processes. In addition, there is the issue of how 
lab activities are structured into work units and organized for 
supervision and management . 

The present study found the task structure mandated by the RFP 
useful in providing more detail on the mission of the labs but. 
tended to be confusing for structuring work. Some kind of 
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hierarchical structuring of programs, projects, and activities was 
used by virtually every lab and seems desirable. Problems arose 
when names and numbers of work units were changed and it became 
difficult to trace a work unit from one report to another (e.g., 
from the full service plan to annual reports, to sel f -assessment 
reports , etc . ) . 



Findings 

Governance and Organization 

The nine regions differed widely in their demographic, 
educational, and economic characteristics. However, there is 
probably as much variation within regions as between them, and 
regional homogeneity is not the basis for the regional organization 
of laboratories. Rather, logistic considerations such as size of 
area , tnav3l d i stances , and number of organ iz at* i ons with which to 
establish relations dictate a sub-national approach to laboratory 
services. Whether nine is the right number or the present 
configuration of states into regions is the right one is an open 
question. Nevertheless, given the difficulty of a laboratory 
establishing itself or changing constituencies, changes should be 
considered only in response to some compel ling demand . It is 
difficult to discern the rationale for the relationship between 
budget levels and regional requirements. It may be desirable to 
find a new formula for the next round of competitions. 

Labs have met federal speci f i cat ions for establ ishing 
independent governing boards and seem well connected to regional 
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constituencies bcth through their boards and adjunct structures 
such as advisory committees. This is especially important given 
the loosely coupled nature of the educational configuration. 

In terms of organization, all labs have some combination of 
program and service units. . Most employ some form of matrix 
management. If they d^d lot have to report to OERI by task, both 
organ izat ional structures and reporting requi rements might be 
simplified. While a small degree of uecentral ization , as practiced 
by FWL and McREL , for example, seems to work reasonably well, the 
more radical form tried initially by NE/I had to be abandoned. 
Labs would probably have to be larger by several orders of 
magnitude before radical decentralization was feasible (i.e. large 
enough to have a "critical mass" at every location). 
Planning 

Some of the planning findings have been covered above in the 
section of planning conceptual frameworks. But a few additional 
comments are in order. 

Labs do a reasonably good job of needs assessment but a poor 
job of regional capacity assessment. Given the requirement that 
labs stress the "with and through" strategy it might be helpful if 
labs viewed regional capacity assessment as measuring the needs of 
regional service improvement organizations. Labs also need to make 
explicit the ways in which components of the strategic planning 
model are analyzed in relations to each other. What is the logic 
and analysis whereby needs are compared with capacity to yield 
planning priorities? Alternatively, if regional capacity 
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assessment is not necessary to good planning it should be dropped 
from the model . 

Self -assessment as presently conducted is trying to serve too 
many purposes, and the process needs to be separated from OERI's 
program evaluation efforts. In add i t i on^produc t testing in the 
development process or selection of promising practices should be 
outside the bounds of self -assessment . Many documentation 
activities (e.g., phone logs) may be useful for internal management 
but are not particularly useful for reporting to OERI . Most 
contract compliance issues should be relegated to quarterly and 
annual reports and limited to a management by exception procedure. 

Most important, new emphasis needs to be placed on an RFP 
requirement that seems to have been largely overlooked. "...the 
laboratory is expected to contribute to knowledge about effective 
strategies for improving education through carefully designed 
studies of how its own dissemination and improvement efforts are 
working" (NIE 1984, p 21). The AEL examination of the study group 
strategy and tracer studies supported by AEL and SEIL appear to be 
examples of what is needed. 

This goal is give explicit recognition in the Task 5 Evaluation 
Collaboration. The Summary Report of the Task 5 Evaluation 
Col laboration (1987) lists as one of four objectives: 

To provide a broadened perspect i ve on laboratory 
operations by synthesizing evaluation information on 
the effectiveness of selected major laboratory 
strategies 
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However , the initial activity identified to implement this 
objective seemed rather limited. It was to compile a descriptive 
synthesis of how Tasks 2 ana 3 were being implemented at each 
laboratory. Only four labs had provided the necessary information 
by the end of 1987. 

Work with Scate Level Decisionmakers 

At 'least one " ab viewed the task of working with state- level 

decisionmakers on school improvement issues* risky because 
maintaining a neutral posture on controversial policy issues was 

difficult, but most labs did not even mention such difficulty. In 
most regions other organizations seem to welcome the idea of a 
lab's convening diverse groups to discuss and analyze issues of 
common interest and otherwise play a catalytic role. 

Labs differed 1n whether they focused their Task 3 effort 
largely on the CSSOs or SEAs more generally, or worked with a 
broader cross section including the governors' officer, legislative 
committees, professional associations and interest groups. They 
also differed considerably in the Improvement role assumed. Some 
functioned primarily as convener of state and regional policy 
forums and conferences , some as resources for pol icy analysis , some 
as monitor of educational reform efforts, etc. This is an area in 
which some progress was made in Task 5 in planning some activities 
for year three and beyond as collaborative lab efforts. Their 
support of William Dunn at the University of Pittsburgh in 
elaborating his ADD model and preparing a handbook of policy 
analysis methods should help considerably in developing a common 
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understanding of the policy analysis function. 

Most labs directed their efforts primarily at the early stages 
of the policy process (agenda and formulation). RBS was a notable 
exception, focusing primarily on the i mpl ementat i or and evaluation 
stages . 

One unresolved issue needs to be addressed. It would seem 

essential for each lab to have information on the educational 

reform legislation and programs in each state and to know just 

which states are engaged in different kinds of initiatives . This 

is logical aspect of environment scanning or needs assessment. In 
addition , *:his information would be useful to many national 

organizations, and organizations participating in the State 
Education Policy Consortium expressed such a need. Some labs 
collect these kinds of data, but whether regional laboratories are 
the best mechanism for developing this kind of national database is 
an open question. Data comparable on a national basis are needed, 
and it might be more practical for OERI or the Center for Education 
Statistics to collect the data in some other way. 
Tasks 2 and 4 

Although Task 2 corresponds to the linkage strategy and Task 4 
corresponds to the improvement strategy, all programs and projects 
classified into and reported to OERI under these rubrics were found 
to have all three strategic dimensions (educational focus, linkage 
strategy, and improvement strategy). In analyzing this set of 
activities it was possible to elaborate various parts of the 
conceptual framework, as reported above. 
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In examining a set of programs to improve administrative 

practices, variations in defining the problem were observed: 

o Local administrators have a great responsibility for 
impl ement ing changes resulting from the reform 
movement . 

o There are important bodies of research on school 
Improvoment , instructional "leadership, etc . , that 
administrators need to become familiar with. 

o Schools and their leaders need to develop a culture 
of Inquiry orienting them toward using knowl edge- 
based resources 1n t he1 r attempts to improvo school s . 

o Scnool administrators are physically Isolated, and 
new structures are needed to put them in touch with 
their peers and help them access knowledge-based 
resources for school Improvement . 

o It 1s a mistake to focus excl uslvely on the 
principal r s role; leadership is provided by a number 
of roles at school and central office levels. 

Two main approaches to improvement were found. One synti "»sizes 
relevant research findings and develops them into training 
packages. The other "collects, reviews, and packages promising 
administrative practices. The most common vehicle for linkage was 
the prlncipalship academy. These academies are sponsored by 
different kinds of service Improvement organizations, and labs 
generally worked "with and through" these organizations. 

The further study of linkage strategies was placed in the 
context of 1nt erorganizat ional theory (N eg and hi 1975), and four 
types of Interoganizat ional collectivities (IC) were formulated, 
based on whether the collectivity was bilateral or multilateral and 
whether it had a single purpose of multiple purposes. These were 
further differentiated in terms of whether the lab was a member o* 
the IC or provided services to a client IC. While it was not 
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possible to make a comprehensive classification of Task 2 and Task 
4 programs, the analysis produced several suggestive findings 
worthy of further study. The collaborative (multi-purpose, 
bilateral) was the least common form of IC. More single-purpose 
arrangements (partnerships and consortia) than multi-purpose 
(col laboratives and networks) were found . 

A more deta i 1 ed exami nat i on of resea rch a pproaches found 
examples of applied research, evaluation, policy stuoies, and 
development. In each of these, examples were found using each 
source of data (primary data, secondary analysis, and interaction 
documentation) . 

A group of programs on staff development was examined to refine 
our understanding of the improvement tactics used. Three labs 
combined a training of trainers approach with development of 
related training packages. These programs tended to be deficient 
in terms of making adequate arrangements for those trained to 
provide second tier training. 

Three other labs combined knowledge production activities such 
as publications and information packages with information 
dissemination activities. Little evidence of impact from these 
strategies was presented. 

Another three labs had unique approaches to staff development. 
FWL worked with the Professional Development Center of the Los 
Angeles Public Schools to develop three sets of training materials 
for mentor teachers. McREL combined network creation and support 
with information dissemination. NWREL used a comprehensive 
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strategy, using just about every improvement and linkage tactic. 

Finally, a separate analysis was made of the "promising 
practices" approach to improvement . This tactic consists of 
collecting descriptions of programs and practices already in 
existence and providing access to this information to others. This 
met hodol ogy was appl ied not onl y to classroom pract ices but al so to 
administrative practices, testing, software, academic standards, 
and professional development programs. Unfortunately, the 
documentation available for this analysis did not provide much 
information on methods of Identifying promising practices, the 
criteria used for selecting them, or evidence of effectiveness. It 
was noted that this approach may or may not be an R&D-based 
approach to Improvement, depending on whether the practice is 
logically consistent with research findings or subjected to a 
rigorous process of evaluation and redesign. 



At a number of points the tension between the rational systems 
perspective and the conf igurational perspective was noted. A 
similar tension exists betwaen requ i rements for proactive planning 
and need for accountability under government contracts on the one 
hand and need to be f 1 exi bl e and res pons i ve to shifting demands 
from constituents on the other. These tensions cannot be 
el1m1n^:ea, but they can be ameliorated if all actors in the drama 
get a better understanding of the limits of planning and 
accountability. Lab programs are high-risk endeavors that can be 
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undercut by unpredictable changes in their environment such as 
changes in SEA personnel . The important thing is to have consensus 
on the broad mission and goals of the labs and recognition that 
changes in specific activities in response to unforeseen 
circumstances are all right if they are consistent with broader 
purposes . The RFP under which these laboratories were sel ected and 
funded did an excellent job of clarifying the mission of the 
program and provides a good basis for program management. It does 
not appear at this point that any radical changes would be needed 
for the next round of competitions. 

Several minor adjustments have been suggested along the way. 
One is to devise a new task structure that does not define any one 
of our three strategic dimensions as a separate task. On the other 
hand, it might be useful to ask thac the three-dimensional schema 
be used in describing each programmatic effort. No program 
description would be complete unless it described its educational 
focus. Improvement strategy, and linkage strategy. 

Another suggestion was to develop more specific guidelines for 
annual reports, quarterly reports, sel f -assessment reports and 
needs sensing reports. There is some overlap among th<sse reports, 
and the separate purpose of each needs sorting out . It wou Id also 
be desirable to define a program structure of programs," projects, 
and activities that would be consistently followed from one report 
to another and over time and that would permit development of a 
management information system for the total laboratory program. 

The regional educational laboratory program has been 
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controversial for over 20 years. 



It never lacks for 



critics who 



ask for hard evidence of massive impacts. Alas, few, if any, 
government programs can meet that test. Perhaps an analogy exists 
between the laboratory program specif ical ly and the educational 
system generally and the bumble bee. We are told that according to 
all the principles of aerodynamics the bumble bee cannot fly - but 
it does . Sometimes it seems as if we are told that the educational 
system is a total failure: that children don't learn, teachers 
don't teach, and new knowledge has no impact on the improvement of 
education. Yet our common sense tells us "it ain T t necessarily so" 
- that* somehow it works in most places and at most times. Our task 
is to find out more about how and why it works when it does so we 
can improve education in all places for all children. The 
laboratories are making steady progress toward doing just that. 
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APPENDIX A 

EXCERPTS FROM REQUEST FOR PROPOSAL 
REGIONAL EDUCATIONAL LABORATORY INSTITUTIONAL OPERATIONS 

Task Structure 

Task 1 : Develop effective governance, management, planning and 
evaluation systems for the laboratory. 

Subtask 1.1: Design and Implement effective governance, 

staffing, and management systems for the laboratory. 

a . Establ 1sh or reshape the govern 1ng board and other 
advisory structures. 

b. Develop or Improve management systems and procedures for 
the laboratory . 

c. Recruit and assign staff and develop or refine policies a 
procedures for their effective utilization. 

Subtask 1.2: Assess regional needs, capabilities, and 

opportunities and establish priorities for laboratory 
activity. 

Subtask 1.3: Prepare plans for future services. 

Subtask 1.4: Conduct self -evaluation of laboratory projects 
and services . 

Task 2 : Work with and through existing prganlzat Ions to Improve 
schools and classrooms. 

Task 3 : Work with State-level decisionmakers on school Improve- 
ment Issues . 

Task Work to create research and development based resources 
for school Improvement . 

Task 5 : Work 1n collaboration with centers and with other 

laboratories on regional and national education problems 

Statements 

1 . Laboratories focus on school and classroom Improvement . 

2. Laboratories feature dissemination and assistance strategies. 



ERLC *U 



APPENDIX 



Page A2 



3. Laboratories engage in applied research and development 
support improvement . 

4 . Laboratories serve designated regions . * 

5. Laboratories have independent governing boards. 

6. Laboratories are part of a nationwide system. 
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APPENOIX 8 

TASK STRUCTURE OF LABORATORY PROGRAMS FOCUSED ON 
IMPROVING ADMINISTRATIVE LEADERSHIP 



The following task structures are taken from proposals, which 
generally contain the most complete statement of rationale and 
task structure. In many cases activities were subsequently 
modified cr deleted. Numbering is that used by each lab: 



o AEL; School Administration and Governance Program . 

1. Provide R&D-based Information to school 
administrators . 

2. Establish and operate State study groups. 

3. Provide technical assistance to school 
administrators through In-State workshops , 
conferences, and SES sites. 

4. Plan and collaborate with other lab programs. 

2 FWL; Professional Preparation and Development 

3.4 Support for school level leadership < 

3.4.1. Refine and demonstrate Peer Assisted 
Leadership (PAL). 

3.4.2. Develop and test a trainer of trainers 
capabl 1 ity . 

3.4.3. Examine special problems in school 
leadership (one topic per year) . 

3.4.4. De/elop and Implement summer leadership 
Institutes for practicing administrators 
and teacher leaders. 

3.4.5. Promote districts* options for school 
Improvement . 

o NE/I : Leadership for School Improvement 

1.1. Support materials for school Improvement 
leadership. 

1.2. The School Improvement Leadership Network 

1.3. Research on leadership for school 
Improvement . 

o NWREL: Rural Networking for Leadership 

o Identify up to three regional agencies that are 

Interested and have good potential as agencies, 
o Develop up to three models for network and 

collaboration development among small ... school s . 
o Develop processes and materials for training 

personnel from cooperating regional agencies and 

schools. 
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o Identify cooperating schools and establish up to 
three networks and test each of the three models 
1n a network . 

o Evaluate the effectiveness of each of three 

model s 1n terms of Improving instruction in the 
schools and prepare the successful model (s) for 
dissemination . 

o Disseminate and support the 1npl ementat ion of 
the successful model (s). 

o R BS; Organizational Effectiveness 

1 . Management development 

2. Secondary school Improvement 

3 . Labor-management cooperat 1on 

o SEDL; Improving Teacher and Administrator Performance 
C.1. Investigating teacher and administrator 

performance-assessment systems associated 
with Implementing rewards, incentives, or 
career- development, 
C.2. Identifying and describing teacher and 
administrator performance-Improvement 
programs that couple performance assessment 
with 1nserv1ce education/staff development. 

o SEIL; C. Improving Leadership Sk1 lis of School 
Principals 

o Provide Information to practitioners on national 
and regional resources; national , State, and 
local Initiatives and current 1 Iterature; and 
form a regional resource network. 

o Design and develop products and staff 

devel opment act1 v1t 1es based upon current 
Council efforts to provide Immediate application 
to school principal training, making n«r of best 
adult learning theories . 

o Collaborate with existing resources to secure 
additional staff development materials. 

o Develop and Implement plans for service 

delivery to practitioners through prlndpalshi p 
academies, with assistance of State advisory 
committees . 

o Provide technical assistance to Intermediate 
service organizations (principal academies)... 

o Plan, Implement and evaluate activities for 

particular application and effect upon 
traditionally underserved populations, and 
revise accordingly. 

o Coordinate with other lab projects to maximize 
effect . 
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o Coordinate with other labs and centers... 

o Identify research needs and recommend a school 

principals research agenda, 
o Evaluate and revise plans to provide 

direction for third, fourth, and fifth years. 
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ADD 

Adm 

AEL 

CIPP 

CCSSO 

CES 

CPRE 

CSSO 

Demo 

DESSI 

Dev 

DM 

ECS 

ERIC 

Eval 

FWL 

GDPA 

IC 

I EL 

IHE 

I OA 

ISA 

KP 



APPENDIX C 
GLOSSARY OF ACRONYMNS AND ABBREVIATIONS 

Analysis, development, and dissemination [model: Dunn] 
Admin Istrator 

Appalachian Educational Laboratory 

Context, Input, program and product [evaluation model] 
Council of Chief State School Officers 
Center for Education Statistics 

Center for Policy Research 1n Education [Rutgers U.] 
Chief state school officer 

# 

Demonstration 

Dissemination Efforts Supporting School Improvement 
(Crandall and Loucks 1983) 

Development 

Decisionmaker 

Educat 1on Commission of the States 

Educa :1onal Resources Information Center 

Evaluation 

Far West Laboratory 

General purpose dissemination assistance (Louis et al 1984) 

I nter organ 1 rational collect 1v1ty 

Institute for Educational Leaders hi p 

Institution of higher education 

I nter organizational arrangement 

Intermediate service agency 

Knowledge production 
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KPO Knowledge production organization 
KPU Knowledge production and utilization 
KU Knowledge utilization 

LEA Local education agency 

McREL M1d Central Regional Educational Laboratory 
NCREL North Central Educational Laboratory 

NE/I The Regional Laboratory for Educational Improvement of the 
Northeast and Islands 

NGA National Governors Association 

NIE National Institute of Education 

NWREL Northwest Regional Educational Laboratory 

OE Office of Education 

OERI Office of Educational Research and Improvement 

RBS Research for Better Schools 

R&D Research and development 

RDD&E Research, development, demonstration, and evaluation 

RDIS Research and Development Interpretation Service 

Res Research 

RS Regional Services [Program] 

RX Reg 1 ona 1 Exchange 

SA Self -assessment 

SEA State education agency 

SEDL Southwest Educational Development Laboratory 

SEP State Education Policy [Program] 

SEIL Southeastern Educational Improvement Laboratory 

SIO Service Improvement organization 
Service delivery organization 
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Tea Teacher 

TT Training of trainers 
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